3 ms·
> training corpus captures the variance inherent in a cognitive system's interactions with the world How do text outputs scraped from the internet + published
by ninininino 2y ago
> training corpus captures the variance inherent in a cognitive system's interactions with the world
How do text outputs scraped from the internet + published works fully represent any existing biological cognitive system's interactions with the world or subjective experience? That's my biggest question I guess.
I'm not sure how text could capture my sensory experiences or my brain's response to them. The bitrate of text feels incredibly low.
- hackinthebochs 2y agoThe point isn't to capture what it's like to be a biological organism, but rather the necessary functional/computational dynamic for sentience. It's plausible that sentience is a class of phenomena, of which biological sentience is but one. An LLM processes tokens represented as vectors embedded in a meaning-space, which is then transformed according to various semantic attributes. This is analogous to how organismal cognition operates on the semantic properties carried by its various sensory states. If cognition represents a kind of functional/computational dynamic over semantic states then learning the dynamic necessary to produce the kinds of texts cognitive systems produce may recover the necessary dynamic for cognition. It certainly won't be equivalent to a biological organism's cognition for the reasons you mention, but the space of cognitive entities may be much broader.