4 ms·
We still seem to be missing an equivalent of explicit memory formation, serializing digested perceptions into working then short term and long term memory. The
by Symmetry 4y ago
We still seem to be missing an equivalent of explicit memory formation, serializing digested perceptions into working then short term and long term memory. The however many thousand tokens in a GPT's buffer can span a much larger span of time than the second's worth of sense impressions your brain can hold without consciousness[1] and memory getting involved but the principle seems to be the same.
This isn't to say that there wouldn't be some simple hack to allow memory formation in chat agents, just that there's at least one advance we need besides simple scale.
[1] As in not subliminal, not anything to do with philosophical notions of qualia.
- lumost 4y agoI haven't seen a good write-up on this, however it appears that large Transformer architectures are directly learning a memory representation in their parameters. ChatGPT can easily return facts, and approximate summaries of facts. Unfortunately ChatGPT is unable to differentiate what it knows, vs. what it thinks it knows. I'd love to see a deep dive into how these models are representing such knowledge.
- amelius 4y agoAlso missing seems to be generalization. Train a robot to kick a white ball, and it will not understand how to kick a green ball.
- readonlybarbie 4y agoHow can we more effectively show a robot that circular objects with a certain position, scale, rotation, mass, color/pattern, or an arbitrary label ("that thing over there") is what we want? Seems like a fun question to solve.