4 ms·
I think the thing missing would be memory. The knowledge of current models is more or less static save for whatever you can cram into their context window. I th
by conception 2y ago
I think the thing missing would be memory. The knowledge of current models is more or less static save for whatever you can cram into their context window. I think if they had memory and thus the ability to learn - “oh hey, I’ve already tried to solve a bug in these ways maybe I won’t get stuck in loop on them!” Would be the agi push for me. Real time incorporating new knowledge into the model is the missing piece.
- casey2 2y agoIt needs some kind of low latency world model. Or at least more agent inspired pretraining (I did x and it failed, I did y and It failed, I should try z now) GOOD (I did x and it failed, I did y and it failed, I should try x now) BAD
- fragmede 2y agoWith MCP/tool use you can tell it to save state into an MD file, simulating this. How much that counts is left as an exercise to the reader.
- conception 2y agoIt actually works really well in my experience. But it eats up context. Using 500-600k token windows per inquiry isn’t cheap.