8 ms·

Anytime I hear someone even hypothesizing with a tin foil hat that some company has developed AGI today in 2023, I hear it the same way I hear someone speculati

Anytime I hear someone even hypothesizing with a tin foil hat that some company has developed AGI today in 2023, I hear it the same way I hear someone speculating about UFOs. It's so absurd that all I can do is shake my head and look around the room hoping to make eye contact with someone sane who is wearing the same expression of raised eyebrows that I am. I could write an essay on why the existence of AGI in 2023 is a comical prospect, but it just seems to be a given to me... It isn't worth the effort because if someone can't see that for themselves already, no amount of explanation from me is going to change their mind. But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain. It isn't a mind. It can't reason. They are no closer to AGI than we were 10 years ago. Chatgpt's text generation capabilities have fooled people into thinking that OpenAI has made substantial progress on creating a reasoning consciousness. But it hasn't. It hasn't at all. And it's so easy to realize that if you interact with the thing.

insanitybit 3y ago
> It isn't worth the effort because if people can't see it for themselves already, no amount of explanation from me is going to change their mind. That's convenient. I mean, if you were to produce such an essay I think it would actually be extremely high value and a lot of people would enjoy reading it. Hell, a decent HN post would be nice. But if you're so unable to express the idea, I frankly question whether you really have such a strong grasp on it.
- windowshopping 3y ago
  Read the last paragraph. It doesn't take an essay.
  - insanitybit 3y ago
    OK, I mean, you used the word essay so I used the word essay. But thanks for editing in that paragraph.
  - stevenhuang 3y ago
    Ah the classic "LLMs are limited bc they can't do character level manipulations". Buddy these are deliberate optimization trade-offs in the tokenizer, LLMs don't even see individual characters. Talk about a red herring. I highly suggest a more refined understanding of LLMs before emphatically stating what they are or aren't. It's frankly embarrassing.
    - oivey 3y ago
      It’s a performance critical optimization that has downsides. It’s a completely fair criticism. It’s like someone claimed a hammer is a universal construction tool, and then someone else pointed out hammers aren’t great for screwing in screws. Your position is that a universal construction tool doesn’t need to work with screws because hammers were never designed to do so.
      - stevenhuang 3y ago
        A hammer is a finished product. LLMs as a technology are in their infancy and see groundbreaking developments every month. There are models trained on character level tokens that handle precisely these issues. https://arxiv.org/abs/2311.06158 https://arxiv.org/abs/2311.06158 https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/training_llms_to_follow_procedure_for_math_gives/ https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/trainin... The fact LLMs can't currently handle character-level changes tell us nothing about whether or not they truly understand meaning. The parent commentator is not probing the right things to answer his question.
aantix 3y ago
But then the Air Force releases video footage of UFO encounters after decades of gaslighting the public? “The government isn’t listening” except Snowden dumps the documents that show 10+ systems designed for mass intelligence?
- stevenhuang 3y ago
  I follow both topics deeply and I'll throw my hat in the ring for speculation, and that is: - there may really be something nonhuman behind the uap phenomenon, then wild implications of what it means to be human, what even is reality, etc, follows - intelligence is simpler an algorithmic process than we thought, consciousness and qualia may be "just" an emergent phenomena
famouswaffles 3y ago
>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. LLMs don't see numbers or letters but tokens. Letter, digit level manipulation is intuitively a hard task. Unsurprisingly, removing that handicap resulted in much better arithmetic abilities. https://arxiv.org/abs/2310.02989 https://arxiv.org/abs/2310.02989 It's pretty telling in my opinion that this is the kind of "proof" people bank hard on. For something that so obviously "does not understand", you'd think providing a task that it would fail that a chunk of humans also wouldn't would be easy.
- insanitybit 3y ago
  Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this: # Reversing the provided string input_string = "473936482738338373926" reversed_string = input_string[::-1] reversed_string
- xyzzy_plugh 3y ago
  How is it intuitively a hard task?
  - mquander 3y ago
    When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.
    - oivey 3y ago
      Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.
mquander 3y ago
> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452dedcf8fc8 https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452ded... > The reversed string of '473936482738338373926' is '629373833837284639374'. So now do you think it understands something?
- CamperBob2 3y ago
  They'll just move the goalpost some more, as usual. God of the Gaps.
- staticman2 3y ago
  That only works if you have it run a python script. If you use the non data analysis version it fails.
  - insanitybit 3y ago
    And? It knows how to formulate and execute that script. There are pretty hardcoded areas of our brain for doing various tasks - when we offload things to those sections are we not really "thinking" ?
    - staticman2 3y ago
      I'm pointing out the disconnect in the conversation: If you say I'm not smart enough to solve a math problem, and I use a python script to solve it written by a smarter person, whether "I" solved it is a matter of interpretation. By not discussing this I felt the above poster was leaving out some key details.
      - insanitybit 3y ago
        I don't agree with the characterization of the script being written by someone else. The model is writing that script, they're just offloading execution of that script to Python.
deleted 3y ago
[deleted]
StarCyan 3y ago
> try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it ChatGPT (4) can accomplish this task easily. Regardless, I'm willing to bet there are at least some humans that can't do this task. And basically all intelligent non-human animals can't either. > And it's so easy to realize that if you interact with the thing. I don't think ChatGPT is AGI, but I think the vast majority of people who have interacted with ChatGPT think that it brings us closer to AGI than what we had 10,20 years ago.
dkarras 3y ago
what is a mind? what does it mean to reason?
qgin 3y ago
The line for AGI used to be the Turing Test. But we blew past that with current models without even blinking and came up with new requirements because we weren’t ready to call something AGI yet.
- keiferski 3y ago
  The Turing Test was one suggestion that many philosophers and computer scientists were/are critical of - and have been critical of for decades.
  - upwardbound 3y ago
    But nonetheless, it was still the most famous test and was specifically set forth as an AGI test by the father of computing. People have been critical of it but there was never any other standard that was as broadly accepted as Turing's proposal.
- lamp987 3y ago
  Not really. Turing test was beaten by much simpler chat bots and long before than ChatGPT (which isn't A(G)I).
  - lostmsu 3y ago
    It really was not, or link to the "beating" result.
    - Jensson 3y ago
      It was in 2014. https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-the-turing-test-by-pretending-to-be-a-13-year-old-boy https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-t... The turing test is a shit test for general intelligence, you can game it by making the AI generate a story that is more engaging than a typical boring human would do. Those stories will get much more human votes than a regular human. Some will go off track and notice it is a dumb program, but those are the minority so on average this dumb bot will pass the turing test.
spaceman_2020 3y ago
I don’t know about AGI but seeing some of the new tools that mix Vision with GPT4 is a little humbling. Like the makeitreal.tldraw.com tool where you can draw a diagram of a game (like Breakout) and it will code up a working game for you.
erkt 3y ago
https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a8283324749d https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a82833... Did you use ChatGPT 3.5 once a year ago or something? The models change often. Edit: most people would struggle to reverse a 20+ digit number if it was listed off to them. This doesn’t make them unintelligent.
- deleted 3y ago
  [deleted]
- golergka 3y ago
  Most people are pretty unintelligent. GPT-4 is already better at most of knowledge work than 50% of human population, it’s not a very high bar.
  - erkt 3y ago
    Unfortunate, but not something I can disagree with.
chpatrick 3y ago
If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too. "Just an incredibly advanced Markov chain" doesn't mean anything. Humans writing text are also incredibly advanced Markov chains.
- usaar333 3y ago
  > If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too. Depends if they were paying attention. Emerging creative behaviors were noticed in GPT-3 in 2020. Image generation was also getting pretty good by 2020 (with Dall-e coming out in early 2021). Very bullish 2020 post on GPT-3: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-post/ https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p...
  - chpatrick 3y ago
    I just said a random date, let's say 2010 then.
- wruza 3y ago
  Writing text isn't that big of a deal either, you just have to be able in winning arguments and converting phonemes to glyphs. I'm not taking sides here, but GPTs defeated me too many times for me to agree with that on better terms.
- batman-farts 3y ago
  No, sorry, I don't buy this assertion when it comes up. Everything I've seen, even from the most advanced image generators, has struck me as a logical follow-on from the "big data" trends of the 2010s. If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel. It also elides the significant encoding of human feedback, a contribution that AI firms have typically been none-too-eager to highlight.
  - 3y ago
keiferski 3y ago
I have the exact same experience and I’m not sure what to make of it. Even repeating the instructions doesn’t work. There is clearly a lack of basic reasoning ability and it can’t comprehend tasks that a five year old would have zero issues with.
blovescoffee 3y ago
Instead of writing an essay, just say concisely what you have in mind here
danielmarkbruce 3y ago
That's fine, but serious people with serious backgrounds in AI like Peter Norvig think we are there, now. It's a debatable topic. https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-... The reason ChatGPT can't reverse numbers is...current tokenization strategies. It's a known shortcoming.
J_cst 3y ago
https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477291879 https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477...
hardwaresofton 3y ago
> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain. I agree with this statement, but why don't I see it described this way more? It really feels like transformers are (large) parameterized markov chains but I never seen anyone describe it this way, is it just not a good approximation/hiding too much of the technical workings to be true?
bbor 3y ago
Define “mind”
ssss11 3y ago
I got it to make a logo for my hypothetical startup yesterday but it put these weird coloured circles beneath it so I asked it to remove those. It generated a whole new logo. No matter how many times I tried to get it to give me the original logo with the changes I wanted it just kept generating new ones. Sometimes with spelling mistakes of the fake company name.. That was not intelligence.
- cabbageears 3y ago
  The DALLE3 and other imagine ones are the worst with those instructions, especially spelling. Like spelling is impossible, but I'm sure they could update it to copy/paste text since current model can't by design. Back when they'd do 4 variants of the image, no matter what you'd say to keep in all 4 of them, it'd go slightly rogue. Very frustrating.
cabbageears 3y ago
"Reverse this sequence of numbers: 473936482738338373926" "The reversed sequence of the numbers 473936482738338373926 is 629373833837284639374." It works like this by using Python. Even when I told it to write a JS function internally to do it and only share the answer, it overruled that. When I asked it to repeat the sequence back in the same order, but starting from the last digit and working backwards, it failed a little bit. It seems like using the language model to pivot to the correct "model" or "logic" to decide to use code is impressive.
deleted 3y ago
[deleted]

insanitybit 3y ago

> It isn't worth the effort because if people can't see it for themselves already, no amount of explanation from me is going to change their mind. That's convenient. I mean, if you were to produce such an essay I think it would actually be extremely high value and a lot of people would enjoy reading it. Hell, a decent HN post would be nice. But if you're so unable to express the idea, I frankly question whether you really have such a strong grasp on it.

windowshopping 3y ago
Read the last paragraph. It doesn't take an essay.
- insanitybit 3y ago
  OK, I mean, you used the word essay so I used the word essay. But thanks for editing in that paragraph.
- stevenhuang 3y ago
  Ah the classic "LLMs are limited bc they can't do character level manipulations". Buddy these are deliberate optimization trade-offs in the tokenizer, LLMs don't even see individual characters. Talk about a red herring. I highly suggest a more refined understanding of LLMs before emphatically stating what they are or aren't. It's frankly embarrassing.
  - oivey 3y ago
    It’s a performance critical optimization that has downsides. It’s a completely fair criticism. It’s like someone claimed a hammer is a universal construction tool, and then someone else pointed out hammers aren’t great for screwing in screws. Your position is that a universal construction tool doesn’t need to work with screws because hammers were never designed to do so.
    - stevenhuang 3y ago
      A hammer is a finished product. LLMs as a technology are in their infancy and see groundbreaking developments every month. There are models trained on character level tokens that handle precisely these issues. https://arxiv.org/abs/2311.06158 https://arxiv.org/abs/2311.06158 https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/training_llms_to_follow_procedure_for_math_gives/ https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/trainin... The fact LLMs can't currently handle character-level changes tell us nothing about whether or not they truly understand meaning. The parent commentator is not probing the right things to answer his question.

aantix 3y ago

But then the Air Force releases video footage of UFO encounters after decades of gaslighting the public? “The government isn’t listening” except Snowden dumps the documents that show 10+ systems designed for mass intelligence?

stevenhuang 3y ago
I follow both topics deeply and I'll throw my hat in the ring for speculation, and that is: - there may really be something nonhuman behind the uap phenomenon, then wild implications of what it means to be human, what even is reality, etc, follows - intelligence is simpler an algorithmic process than we thought, consciousness and qualia may be "just" an emergent phenomena

famouswaffles 3y ago

>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. LLMs don't see numbers or letters but tokens. Letter, digit level manipulation is intuitively a hard task. Unsurprisingly, removing that handicap resulted in much better arithmetic abilities. https://arxiv.org/abs/2310.02989 https://arxiv.org/abs/2310.02989 It's pretty telling in my opinion that this is the kind of "proof" people bank hard on. For something that so obviously "does not understand", you'd think providing a task that it would fail that a chunk of humans also wouldn't would be easy.

insanitybit 3y ago
Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this: # Reversing the provided string input_string = "473936482738338373926" reversed_string = input_string[::-1] reversed_string
xyzzy_plugh 3y ago
How is it intuitively a hard task?
- mquander 3y ago
  When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.
  - oivey 3y ago
    Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.

mquander 3y ago

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452dedcf8fc8 https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452ded... > The reversed string of '473936482738338373926' is '629373833837284639374'. So now do you think it understands something?

CamperBob2 3y ago
They'll just move the goalpost some more, as usual. God of the Gaps.
staticman2 3y ago
That only works if you have it run a python script. If you use the non data analysis version it fails.
- insanitybit 3y ago
  And? It knows how to formulate and execute that script. There are pretty hardcoded areas of our brain for doing various tasks - when we offload things to those sections are we not really "thinking" ?
  - staticman2 3y ago
    I'm pointing out the disconnect in the conversation: If you say I'm not smart enough to solve a math problem, and I use a python script to solve it written by a smarter person, whether "I" solved it is a matter of interpretation. By not discussing this I felt the above poster was leaving out some key details.
    - insanitybit 3y ago
      I don't agree with the characterization of the script being written by someone else. The model is writing that script, they're just offloading execution of that script to Python.

deleted 3y ago

[deleted]

StarCyan 3y ago

> try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it ChatGPT (4) can accomplish this task easily. Regardless, I'm willing to bet there are at least some humans that can't do this task. And basically all intelligent non-human animals can't either. > And it's so easy to realize that if you interact with the thing. I don't think ChatGPT is AGI, but I think the vast majority of people who have interacted with ChatGPT think that it brings us closer to AGI than what we had 10,20 years ago.

dkarras 3y ago

what is a mind? what does it mean to reason?

qgin 3y ago

The line for AGI used to be the Turing Test. But we blew past that with current models without even blinking and came up with new requirements because we weren’t ready to call something AGI yet.

keiferski 3y ago
The Turing Test was one suggestion that many philosophers and computer scientists were/are critical of - and have been critical of for decades.
- upwardbound 3y ago
  But nonetheless, it was still the most famous test and was specifically set forth as an AGI test by the father of computing. People have been critical of it but there was never any other standard that was as broadly accepted as Turing's proposal.
lamp987 3y ago
Not really. Turing test was beaten by much simpler chat bots and long before than ChatGPT (which isn't A(G)I).
- lostmsu 3y ago
  It really was not, or link to the "beating" result.
  - Jensson 3y ago
    It was in 2014. https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-the-turing-test-by-pretending-to-be-a-13-year-old-boy https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-t... The turing test is a shit test for general intelligence, you can game it by making the AI generate a story that is more engaging than a typical boring human would do. Those stories will get much more human votes than a regular human. Some will go off track and notice it is a dumb program, but those are the minority so on average this dumb bot will pass the turing test.

spaceman_2020 3y ago

I don’t know about AGI but seeing some of the new tools that mix Vision with GPT4 is a little humbling. Like the makeitreal.tldraw.com tool where you can draw a diagram of a game (like Breakout) and it will code up a working game for you.

erkt 3y ago

https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a8283324749d https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a82833... Did you use ChatGPT 3.5 once a year ago or something? The models change often. Edit: most people would struggle to reverse a 20+ digit number if it was listed off to them. This doesn’t make them unintelligent.

deleted 3y ago
[deleted]
golergka 3y ago
Most people are pretty unintelligent. GPT-4 is already better at most of knowledge work than 50% of human population, it’s not a very high bar.
- erkt 3y ago
  Unfortunate, but not something I can disagree with.

chpatrick 3y ago

If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too. "Just an incredibly advanced Markov chain" doesn't mean anything. Humans writing text are also incredibly advanced Markov chains.

usaar333 3y ago
> If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too. Depends if they were paying attention. Emerging creative behaviors were noticed in GPT-3 in 2020. Image generation was also getting pretty good by 2020 (with Dall-e coming out in early 2021). Very bullish 2020 post on GPT-3: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-post/ https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p...
- chpatrick 3y ago
  I just said a random date, let's say 2010 then.
wruza 3y ago
Writing text isn't that big of a deal either, you just have to be able in winning arguments and converting phonemes to glyphs. I'm not taking sides here, but GPTs defeated me too many times for me to agree with that on better terms.
batman-farts 3y ago
No, sorry, I don't buy this assertion when it comes up. Everything I've seen, even from the most advanced image generators, has struck me as a logical follow-on from the "big data" trends of the 2010s. If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel. It also elides the significant encoding of human feedback, a contribution that AI firms have typically been none-too-eager to highlight.
- 3y ago

keiferski 3y ago

I have the exact same experience and I’m not sure what to make of it. Even repeating the instructions doesn’t work. There is clearly a lack of basic reasoning ability and it can’t comprehend tasks that a five year old would have zero issues with.

blovescoffee 3y ago

Instead of writing an essay, just say concisely what you have in mind here

danielmarkbruce 3y ago

That's fine, but serious people with serious backgrounds in AI like Peter Norvig think we are there, now. It's a debatable topic. https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-... The reason ChatGPT can't reverse numbers is...current tokenization strategies. It's a known shortcoming.

J_cst 3y ago

https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477291879 https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477...

hardwaresofton 3y ago

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain. I agree with this statement, but why don't I see it described this way more? It really feels like transformers are (large) parameterized markov chains but I never seen anyone describe it this way, is it just not a good approximation/hiding too much of the technical workings to be true?

bbor 3y ago

Define “mind”

ssss11 3y ago

I got it to make a logo for my hypothetical startup yesterday but it put these weird coloured circles beneath it so I asked it to remove those. It generated a whole new logo. No matter how many times I tried to get it to give me the original logo with the changes I wanted it just kept generating new ones. Sometimes with spelling mistakes of the fake company name.. That was not intelligence.

cabbageears 3y ago
The DALLE3 and other imagine ones are the worst with those instructions, especially spelling. Like spelling is impossible, but I'm sure they could update it to copy/paste text since current model can't by design. Back when they'd do 4 variants of the image, no matter what you'd say to keep in all 4 of them, it'd go slightly rogue. Very frustrating.

cabbageears 3y ago

"Reverse this sequence of numbers: 473936482738338373926" "The reversed sequence of the numbers 473936482738338373926 is 629373833837284639374." It works like this by using Python. Even when I told it to write a JS function internally to do it and only share the answer, it overruled that. When I asked it to repeat the sequence back in the same order, but starting from the last digit and working backwards, it failed a little bit. It seems like using the language model to pivot to the correct "model" or "logic" to decide to use code is impressive.

deleted 3y ago

[deleted]