Dran

my guess is something like this

https://330k.github.io/misc_tools/unicode_steganography.html

A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.

Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.

You are definitely correct though that it’d be trivial to detect and strip by someone aware of it.

Not necessarily, there are valid “characters” that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.

I assume that’s what they’re talking about here

Here’s an example of a steganography encoding technique using this method. https://330k.github.io/misc_tools/unicode_steganography.html

You can’t just write off capital expenditure though. The hardware, even for “effecient” MOE inference is still very expensive to buy, house, run, and cool. Even assuming open-weight model serving at $0 r&d for the models themselves, mixing high-prefill workloads doesn’t batch well with decode heavy concurrency (or other prefill-heavy jobs). The moment you do anything nontrivial you start running into very complicated architectural problems to efficiently solve at scale.

Hardware that is useful for 5-10 years at most, plus development and support for the inference workflows, doesn’t leave a lot of margin on the table.

My gut, along with basically everything I read, suggests that not most (even pure inference) shops are not profitable and are still floating on loans or vc money.

Can you cite your source on the claim that “inference is currently insanely profitable”? Everything I read suggests that openai and anthropic lose money on their plans.