7 ms·
> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and
by dragonwriter 3y ago
> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks.
I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens that correspond to whole words not sequences of letters, so a game that involves dealing with sequences of letters constrained to those that are valid words doesn’t match the way they process information?
> Providing an LLM with examples and step-by-step instructions in a prompt means the user is figuring out the “reasoning steps” and handing them to the LLM, instead of the LLM figuring them out by itself. We have “reasoning machines” that are intelligent but seem to be hitting fundamental limits we don’t understand.
But providing examples with different, contextually-appropriate sets of reasoning steps results can enable the model to choose its own, more-or-less appropriate, set of reasoning steps for particular questions not matching the examples.
> It’s unclear if better prompting and bigger models using existing attention mechanisms can achieve AGI.
Since there is no objective definition of AGI or test for it, there’s no basis for any meaningful speculation on what can or cannot achieve it; discussions about it are quasi-religious, not scientific.
- jncfhnb 3y agoI think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.
- pixl97 3y agoA consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.
- jncfhnb 3y agoI would argue it needs to be at least somewhat continuous. Perhaps discrete on some granularity but if something is just a function waiting to be called it’s not an intelligent entity. The entity is the calling itself.
- haswell 3y agoHuman consciousness does though, e.g. the flow state. F1 drivers are a good example. We tend to not experience continuous time because we repeatedly get distracted by our thoughts, but entering the continuous stream of now is possible with practice and is one of the aims of many meditators.
- wongarsu 3y agoYou could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would qualify as an AGI as we currently define it. Given a sufficiently good LLM with good reasoning capabilities such a setup might be able to It would be able to do many of the things we currently expect AGIs to be able to do (given a sufficiently good LLM with good reasoning capabilities), including planning and learning new knowledge and new skills (by collecting and storing positive and negative examples in its "memory"). But its learning would be limited, and I'm sure as soon as it exists we would agree that it's not AGI
- lgas 3y agoIt works better to give it access to functions to call for actions and remembering stuff, but this approach does provide some interesting results.
- sophiabits 3y agoThis already exists (in a slightly different prompt format); it's the underlying idea behind ReAct: https://react-lm.github.io https://react-lm.github.io As you say, I'm skeptical this counts as AGI. Although I admit that I don't have a particularly rock solid definition of what _would_ constitute true AGI.
- marquisdepolis 3y ago(Author here). I tried creating something similar in order to solve wordle etc, and the interesting part is that it is insufficient still. That's part of the mystery.
- PopePompus 3y agoI don't understand why you believe it must experience continuous time. If you had a system which clearly could reason, which could learn new tasks on its own, which didn't hallucinate any more than humans do, but it was only active for the period required for it to complete an assigned task, and was completely dormant otherwise, why would that dormant period disqualify it as AGI? I agree that such a system should probably not be considered conscious, but I think it's an open question whether or not consciousness is required for intelligence.
- kaibee 3y agoI think its note worthy that humans actually fail this test... We have to go dormant for 8 hours every day.
- Hunpeter 3y agoYes, but our brain is still working and processing information at those times as well, isn't it? Even if not in the same way as it does when we're conscious.
- PopePompus 3y agoWhat about general anesthesia? I had a major operation during which most of my brain was definitely offline for at least 8 hours.
- adrianN 3y agoI try my best not to experience continuous time for at least eight hours a day.
- jncfhnb 3y agoThen for at least eight hours a day you don’t qualify as a generally intelligent system.
- card_zero 3y agoIf I spend some amount of the day bathing, some amount of it scratching, some amount of it thinking vaguely about racoons without any clear conclusions, and a lot of it drinking tea, I wonder how many seconds remain during which I qualified as generally intelligent.
- jncfhnb 3y agoI feel you qualify during all of those waking seconds
- card_zero 3y agoRacoons are said to be intelligent because they're good at opening locks. On the other hand, when they have food and are within ten feet of a pool of water, they will dip the food in the water and rub it between their paws for no reason. They can reason about the locks, but not about the food. Meanwhile, I in theory can reason about anything, but in practice I wouldn't count on it. Whereas an LLM can't reason, but it's very sharp and always ready to react appropriately.
- jncfhnb 3y agoI’m not familiar with this raccoon behavior but it sure doesn’t sound like it’s done without reason. An LLM is never ready to react to anything because it’s just a matrix that needs a higher level system to invoke it.
- naasking 3y agoSome good prompt-reply interactions are probably fed back in to subsequent training runs, so they're still stateful/have memory in a way, there's just a long delay.
- jncfhnb 3y agoThat’s not the AGI’s state. That’s just some past information.
- naasking 3y agoState is a function of accumulated past information.
- jncfhnb 3y agoState is a function of accumulated past. That does not mean that having some past written down makes you stateful. A stateful thing has to incorporate the ongoing changes.
- naasking 3y agoWhich is what I described: some successful prompt-replies are fed back into subsequent training runs.
- jncfhnb 3y agoNo… that implies the model never has active state and is being replaced with a different, stateless model. This is similar to the difference between Actor.happy = True And Actor = happier(Actor)
- naasking 3y agoBoth of your examples are stateful systems from the outside, given a suitable choice of timeframe, the latter one is just how purely functional systems represent state. Theoretically they can simulate each other, and the endpoint you use to access Actor will still reference the latest Actor. The only reason you're calling them different is because you insist on using a specific timeframe to exclude considering one as stateful, and I'm pointing out that that isn't strictly necessary.
- layer8 3y agoRegarding Wordle, it should be straightforward to make a token-based version of it, and I would assume that that has been tried. It seems the obvious thing to do when one is interested in the reasoning abilities necessary for Wordle.
- deleted 3y ago[deleted]
- furyofantares 3y agoThat doesn't seem straightforward - although it's blind to letters because all it sees are tokens, it doesn't have much training data ABOUT tokens.
- baobabKoodaa 3y agoWhat parent is saying is that instead of asking the LLM to play a game of Wordle with tokens like TIME,LIME we ask it to play with tokens like T,I,M,E,L. This is easy to do.
- furyofantares 3y agoAnd if you tell it to think up a word that has an E in position 3 and an L that's somewhere in the word but not in position 2, it's not going to be any better at that if you tell it to answer one letter at a time.
- layer8 3y agoThe idea is, instead of five-letter-words, play the game with five-token-words.
- furyofantares 3y agoThat was my original interpretation, and while all it sees are tokens, roughly none of its training data is metadata about tokenizing. It knows far less about the positions of tokens in words than it does about the positions of letters in words.
- rainsford 3y agoArriving at a generally accepted scientific definition of AGI might be difficult, but a more achievable goal might be to arrive at a scientific way to determine something is not AGI. And while I'm not an expert in the field, I would certainly think a strong contender for relevant criteria would be an inability to process information in a way other than the one a system was explicitly programmed to, even if the new way of processing information was very related to the pre-existing method. Most humans playing Wordle for the first time probably weren't used to thinking about words that way either, but they were able to adapt because they actually understand how letters and words work. I'm sure one could train an LLM to be awesome at Wordle, but from an AGI perspective the fact that you'd have to do so proves it's not a path to AGI. The Wordle dominating LLM would presumably be perplexed by the next clever word game until trained on thinking about information that way, while a human doesn't need to absorb billions of examples to figure it out. I was originally pretty bullish on LLMs, but now I'm equally convinced that while they probably have some interesting applications, they're a dead-end from a legitimate AGI perspective.
- lukan 3y ago"they're a dead-end from a legitimate AGI perspective" Or another piece of the puzzle to achieve it. It might not be one true path, but a clever combination of existing working pieces where (different) LLMs are one or some of those pieces. I believe there is also not only one way of thinking in the human brain, but my thought processes happen on different levels and maybe based on different mechanism. But as far as I know, we lack details.
- JoshuaDavid 3y agoWhat about an LLM that can't play wordle itself without being trained on it, but can write and use a wordle solver upon seeing the wordle rules? I think "can recognize what tools are needed to solve a problem, build those tools, and use those tools" would count as a "path to AGI".
- Al-Khwarizmi 3y agoAn LLM doesn't even see individual letters at all, because they get encoded into tokens before they are passed as input to the model. It doesn't make much sense to require reasoning with things that aren't even in the input as a requisite for intelligence. That would be like an alien race that could see in an extra dimension, or see the non-visible light spectrum, presenting us with problems that we cannot even see and saying that we don't have AGI when we fail to solve them.
- nathan_compton 3y ago"Since there is no objective definition of AGI or test for it, there’s no basis for any meaningful speculation on what can or cannot achieve it; discussions about it are quasi-religious, not scientific." This is such a weird thing to say. Essentially _all_ scientific ideas are, at least to begin with, poorly defined. In fact, I'd argue that almost all scientific ideas remain poorly defined with the possible exception of _some_ of the basic concepts in physics. Scientific progress cannot be and is not predicated upon perfect definitions. For some reason when the topic of consciousness or AGI comes up around here, everyone commits a sort of "all or nothing" logical fallacy: absence of perfect knowledge is cast as total ignorance.
- eru 3y agoYes. That absence of perfect definition was part of why Turing came with his famous test so long ago. His original paper is a great read!
- Etherlord87 3y agoSam Harris argues similarly in The Moral Landscape. There's this conception objective morality cannot exist outside of religion, because as soon as you're trying to prove one, philosophers rush with pedantic criticism that would render any domain of science invalid.
- nathan_compton 3y agoI kinda get where Sam Harris is coming from, but its kind of silly to call what he is talking about morality. As far as I can tell, Harris is just a moral skeptic who believes something like "we should get a bunch of people together to decide kind of what we want in the world and then rationally pursue those ends." But that is very different from morality as it was traditionally understood (eg, facts about behaviors which are objective in their assignment of good and bad).
- Eisenstein 3y agoWhat is the rough definition, then?