So, the excitement of using the newly released FT230X for serial over USB, turned to a headache:
One is the original and one is a copy "borrowed" from a cheep USB to Serial cable. The message was 0xAA, but all I got was zeros.
Same problem with live data from the board. Annoying little bug!
Turns out, that revision B & C (one of which I have), still has some kinks in it. In my case, opening the virtual COM port too quickly caused the FT230X chip to start sending garbage. Though to be fair, this particular case might be the drivers fault, since it worked a lot more reliably on another machine with an older kernel.
The solution, wait 20-30 seconds after plugging the board in to the computer before opening a connection.
But the good news is, over-clocking the ATmega328 from 16MHz to 20MHz works like a charm!