4 ms·
Yes. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output and/or further c
by optimalsolver 2y ago
Yes.
An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output and/or further compress its sensory history.
An LLM is doing exactly nothing while waiting for the next prompt.
- m3kw9 2y agoWe are thinking machines and we keep thinking because we have one goal which is to survive, machines have no such true goals. I mean true because our biology forces us to do that
- xvector 2y agoIs a human with short term memory loss - or otherwise unable to improve their skills - generally intelligent?
- optimalsolver 2y agoWould you let such a person handle an important task for you?
- rcarmo 2y agoA human with short term memory loss still has agency and impatience.
- xvector 2y agoAgency is essentially solved, we don't enable it in common models because of "safety" Is impatience a requirement for general intelligence? Why?
- aetherson 2y agoThere is a limited amount of computation that you can useful do in the absence of new input (like an LLM between prompts). If you do as much computation as you usefully can (with your current algorithmic limits) in a burst immediately when you receive a prompt, output, and then go into a sleep state, that seems obviously better than receive a prompt, output, and then do some of the computation that you can usefully do after your output.
- amelius 2y agoCan't we just finetune the model based on the LLM's output? Has anyone tried it?
- soulofmischief 2y agoNot only does a training pass take more time and memory than an inference pass, but if you remember the Microsoft Tay incident, it should be self-explainatory why this is a bad idea without a new architecture.
- alchemist1e9 2y agoself prompting via chain of thought and tree of thought can be used in combination with updating memory containing knowledge graphs combined with cognitive architectures like SOAR and continuous external new information and sensory data … with LLM at the heart of that system and it will exactly be a “thinking machine”. The problem is currently it’s very expensive to be continuously running inference full time and all the engineering around memory storage, like RAG patterns, and the cognitive architecture design is all a work in progress. It’s coming soon though.
- whatwhaaaaat 2y agoWe’re going to need to see this working. From my perspective many of the corporate llms are actually getting worse. Slop feedback loops. By no means has it been proven that llms functioning the way you describe will result in superior output.
- Uehreka 2y agoI see people say this all the time and it sounds like a pretty cosmetic distinction. Like, you could wire up an LLM to a systemd service or cron job and then it wouldn’t be “waiting”, it could be constantly processing new inputs. And some of the more advanced models already have ways of compressing the older parts of their context window to achieve extremely long context lengths.
- Earw0rm 2y agoIf it's coalescing learning in realtime across all user/sessions, that's more constant than you're maybe giving it credit for. I'm not sure if GPT4o and friends are actually built that way though.
- wat10000 2y agoIf you had a magic stasis tube that kept a person suspended while you weren’t talking to them, they’d still be a thinking machine.