3 ms·
LLMs don't "understand bitwise noise at a native level". The unit of perception for an LLM is a token, a soundly superbit level. They would have the same diffic
by scott_weber 27d ago
LLMs don't "understand bitwise noise at a native level". The unit of perception for an LLM is a token, a soundly superbit level. They would have the same difficulty with bits as the number of "r"s in "strawberry". Yes they can be post trained to deal with such difficulties, but it's no more "native" than a human who memorised the ASCII table.
- ethbr1 26d agoI struggle to reconcile frontier LLMs' proven decompilation abilities with any assertion that humans possess equivalent parsing skills. They're looking at information as information; we're looking at it in some higher order abstraction.
- ETH_start 25d agoThe degree of nativeness of the skill is orthogonal to the raw power of these systems. A frontier model has access to dozens of GPUs, each moving terabits of data, and performing trillions of matrix multiplication operations, per second.
- shmeeed 26d agoThis does refute the bitwise aspect, but not the possibility of steganography via other methods.