4 ms·
I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory m
by aithrowawaycomm 2y ago
I think the authors are using information theory to inappropriately flatten the complexity of the problem. On one hand we have “bits” of pre-processed sensory measurement data, then on the other hand we have “bits” of post-processed symbolic data: in many cases directly so via human language, but that would also include “the Terran unit moved a short distance” as a compact summary of a bunch of pixels updating in StarCraft. This even extends to the animal examples: the 10 bits/s figure applies to higher-level cognition. The crucial difference is that the sensory bits can be interpreted via the same “algorithm” in a context-independent way, whereas the higher-level cognition bits need their algorithms chosen very carefully (perhaps being modified at runtime).
So I am just not sure why 10 bits/s of symbolic data processing is especially slow in the first place. We don’t have a relevant technological comparison because none of our technology actually processes data in that fashion.
- nis0s 2y agoWhen compared directly to the 10^9 bits/s for sensory information, which uses the same type of information, it is slow.
- aeonik 2y agoIt's more like quantum information theory isn't it? https://en.wikipedia.org/wiki/Quantum_information https://en.wikipedia.org/wiki/Quantum_information
- kbelder 2y agoI'm running a chat LLM on my local pc. It spits out text just slightly faster than I can type, but it is using much of my CPU and redlining my GPU. Is it processing at a dozen bits per second, or hundreds of millions? If the text the LLM generates is "that is true", can I consider that one bit of information? I agree, they're artificially simplifying the framing of the question to generate a lower number than is sensible.