4 ms·
At this point the thing holding ChatGPT back from universally-accepted "AGI" label is its few remaining sub-human skillsets, like forgetting things from too man
by gryfft 3y ago
At this point the thing holding ChatGPT back from universally-accepted "AGI" label is its few remaining sub-human skillsets, like forgetting things from too many tokens ago that a human would not forget.
My prediction is that over the course of the next 6-48 months, we'll see the emergence of LLMs with "working memory," "short term memory," and "long term memory," with working memory being more or less current LLM capabilities, short term memory being made up of a fast one-shot summarization which then gets temporarily stored raw on disk, and long-term storage getting transcribed into a LORA-like module overnight based on perceived importance of the short term memories.
I think emotion analogues will be important for the last part, as emotion processing plays a big role in memory formation (this is an adaptation: we more strongly remember things that we had strong emotions about because they're more important to us.)
So, 6-48 months to computer systems that feel (/have emotion analogue) and sleep to dream (/summarize into long-term storage overnight.)
Those developments, I'm confident, will absolutely silence anyone who says it's not "real" AGI. But then, at that point, you can potentially have built a being that can have feelings about its own existence, and then things get Interesting.
- philippta 3y agoThanks a lot for that detailed explanation. That makes perfect sense. I wonder if all these additional capabilities can be bolted onto existing LLMs or need another iteration to the transformer architecture plus retraining.
- ilaksh 3y agoLLMs are not the end of AI research. See LMM (large multimodal model), cognitive architecture, spiking neural networks, I-JEPA, etc. There are going to be multiple totally different types of AI that may be called be AGI, depending on who you ask. For emotions, see Pei Wang's research.
- latexr 3y ago> That makes perfect sense. On the contrary, it has multiple glaring flaws. When you consider that you can ask an LLM one question and get a nonsensical answer, saying that stopping them from forgetting will bring them closer to “universally-accepted” AGI has no basis in reality. Humans can’t even universally agree that the Earth is not flat, it is a pipe dream to think LLMs will bring any consensus in a few months. Another poster points out other issues with the answer: https://news.ycombinator.com/item?id=37915367 https://news.ycombinator.com/item?id=37915367
- Someone 3y ago> At this point the thing holding ChatGPT back from universally-accepted "AGI" label is its few remaining sub-human skillsets How do we know there are a few remaining ones to reach AGI and not zillions? How do we know adding “memory” is sufficient for it to acquire them? I think the answers to those questions are “we don’t have the faintest idea” and “we know it won’t be enough”. As to the latter, one thing LLMs cannot do that we think is essential for human intelligence is to think logically. They can regurgitate logic from memory, but they can’t come up with correct original ideas, unless by accident.
- beanbean01 3y agoI suspect that a great many of the people who say AGI is just around the corner and are giving <2030 timelines are saying that because they want it to be around the corner.
- latexr 3y agoJust like self-driving cars, we’re likely to hear people say “AGI is just around the corner” for an unreasonable amount of time.