4 ms·

Pretty much. Capabilities we now consider mundane were science fiction just three years ago, as far as anyone not employed by OpenAI was concerned. We already

by buu700 1y ago

Pretty much. Capabilities we now consider mundane were science fiction just three years ago, as far as anyone not employed by OpenAI was concerned. We already had a definition of AGI. We hit it. Are you sure about that? Which definition are you referring to? From what I can tell with Google and Grok, every proposed definition has been that AGI strictly matches or exceeds human cognitive capabilities across the board. Generative AI is great, but it's not like you could just assign an arbitrary job to a present-day LLM, give it access to an expense account, and check in quarterly with reasonable expectations of useful progress.

wild_egg 1y ago
You generally can't just have a quarterly check-in with humans either. There's a significant fraction of humanity that would not clear the bar to meet current AGI definitions. The distribution of human cognitive abilities is vast and current AI systems definitely exceed the capabilities of a surprising number of people.
- buu700 1y ago
  That's fair. I suppose it gets at the heart of the question of what the specific threshold for AGI should be. Is it matching or exceeding all cognitive capabilities of any given human, an average 100-IQ adult, a severely learning-disabled child, or something else entirely? Maybe AGI is in the eye of the beholder, and a given person just has to decide by using their own experience of human intelligence as a benchmark. In other words, AGI is AGI whenever majority consensus decides it's a universal replacement for ordinary human intelligence.
ninetyninenine 1y ago
>We already had a definition of AGI. We hit it. The turing test.
- rmunn 1y ago
  The Turing test was never a test of thinking: Turing said that thinking was difficult to define and so he decided to "replace the question by another, which is closely related to it" (I disagree with him there) "and is expressed in relatively unambiguous words," i.e. the question of whether a chatbot can fool a text-only observer into thinking it's human. Clearly, current LLMs have passed the Turing test, as witnessed by the difficulty many schools have in enforcing "do not use LLMs to do your homework" rules. But even Turing didn't say his test was a test of intelligence, just a test "closely related" to intelligence. And if he had seen today's LLMs, I think he would have revised that opinion, because today's LLMs generate text with no underlying fact model, no fundamental understanding of the truth behind the words they're saying. (No understanding, even, of the concepts of truth or falsehood). I think today's LLMs have demonstrated that being able to string words together in coherent sentences is not "closely related" to intelligence at all.
- buu700 1y ago
  As far as I can tell, passing the Turing test has never been the majority-consensus definition of AGI. It seems to me that the Turing test has fundamentally always been about proving a negative: if something fails the Turing test, it's probably not AGI. For reference, the term AGI post-dates the Turing test by half a century. I also don't personally remember ever hearing the exact term "artificial general intelligence" prior to 2023 or 2024, or at least it wasn't mainstream the way it is today. If AGI had truly ever been defined by the Turing test, then Cleverbot should've been hailed as AGI when it passed the test in 2011. Even if we did all agree to call it that, we'd still need some other term for what we actually mean when we say "AGI" today. Cleverbot-era chatbots were cute toys, but they weren't capable of doing useful work of any kind.
  - ninetyninenine 1y ago
    That’s not accurate. The Turing test was always intended as a benchmark for general intelligence. Turing’s 1950 paper explicitly proposed it as a way to operationalize the question “Can machines think?” not as a parlor trick about conversation but as a proxy for indistinguishability in intellectual behavior. The whole point of the imitation game was to sidestep metaphysical arguments and reduce intelligence to functional equivalence. If a machine could consistently hold its own in unrestricted dialogue, it would demonstrate the breadth, adaptability, and contextual understanding that characterize general intelligence. The term AGI may have come later, but the concept it represents traces directly back to Turing’s framing. When early AI researchers talked about “strong AI” or “thinking machines,” they were using the same conceptual lineage. The introduction of the acronym doesn’t rewrite that history, it just gave a modern label to an old idea. The Turing test was never meant to detect a “negative” but to give a concrete, falsifiable threshold for when positive claims of general intelligence might be justified. As for Cleverbot, it never truly passed the test in any rigorous or statistically sound sense. Those 2011 headlines were based on short exchanges with untrained judges and no control group. Passing a genuine Turing test requires sustained coherence, reasoning across domains, and the ability to handle novel input gracefully. Cleverbot couldn’t do any of that. It failed the spirit of the test even if it tricked a few people in the letter of it. By contrast, modern large language models can pass the Turing test in flying colors. They can maintain long, open-ended conversations, reason about complex subjects, translate, summarize, and solve problems across many domains. Most human judges would be unable to tell them apart from people in text conversation, not for a few sentences but for hours. Granted, one can often tell ChatGPT is an AI because of its long and overly descriptive replies, but that’s a stylistic artifact, not a limitation of intelligence. The remarkable thing is that you can simply instruct it to imitate casual human conversation, and it will do so convincingly, adjusting tone, rhythm, and vocabulary on command. In other words, the test can be passed both intentionally and effortlessly. The Turing test was never obsolete; we finally built systems that can truly meet it.
ninetyninenine 1y ago
>Generative AI is great, but it's not like you could just assign an arbitrary job to a present-day LLM, give it access to an expense account, and check in quarterly with reasonable expectations of useful progress. Has anyone tried this yet?
- jdlshore 1y ago
  There have been a few attempts at “building a business by following ChatGPT’s instructions.” All the ones I’m aware of failed, and were really just a scheme to grab eyeballs. There’s also a website showing a company’s attempt to use multiple LLMs to close its (real-world, but historical) accounting month-end books. They fail miserably with compounding errors.
- buu700 1y ago
  I'd be interested in seeing the results if so. Like someone with $500k to burn just handing it to an AI and telling it to found a startup, including incorporation, vibe coding and deploying a product, publishing a website, bringing on cofounders and/or advisors and/or employees as needed, fundraising, etc. My guess is finding agentic tooling that could even push the model hard enough to attempt that in earnest would be a problem, and the compute bills would add up quickly, but it would be a very interesting experiment. Probably with hilarious results, but seeing what the actual failure points are would be useful information. Maybe it would actually end up hiring and paying someone to do something insanely illegal.
- famouswaffles 1y ago
  Vending Bench ? https://andonlabs.com/evals/vending-bench https://andonlabs.com/evals/vending-bench

by buu700 1y ago

wild_egg 1y ago

You generally can't just have a quarterly check-in with humans either. There's a significant fraction of humanity that would not clear the bar to meet current AGI definitions. The distribution of human cognitive abilities is vast and current AI systems definitely exceed the capabilities of a surprising number of people.

buu700 1y ago
That's fair. I suppose it gets at the heart of the question of what the specific threshold for AGI should be. Is it matching or exceeding all cognitive capabilities of any given human, an average 100-IQ adult, a severely learning-disabled child, or something else entirely? Maybe AGI is in the eye of the beholder, and a given person just has to decide by using their own experience of human intelligence as a benchmark. In other words, AGI is AGI whenever majority consensus decides it's a universal replacement for ordinary human intelligence.

ninetyninenine 1y ago

>We already had a definition of AGI. We hit it. The turing test.

rmunn 1y ago
The Turing test was never a test of thinking: Turing said that thinking was difficult to define and so he decided to "replace the question by another, which is closely related to it" (I disagree with him there) "and is expressed in relatively unambiguous words," i.e. the question of whether a chatbot can fool a text-only observer into thinking it's human. Clearly, current LLMs have passed the Turing test, as witnessed by the difficulty many schools have in enforcing "do not use LLMs to do your homework" rules. But even Turing didn't say his test was a test of intelligence, just a test "closely related" to intelligence. And if he had seen today's LLMs, I think he would have revised that opinion, because today's LLMs generate text with no underlying fact model, no fundamental understanding of the truth behind the words they're saying. (No understanding, even, of the concepts of truth or falsehood). I think today's LLMs have demonstrated that being able to string words together in coherent sentences is not "closely related" to intelligence at all.
buu700 1y ago
As far as I can tell, passing the Turing test has never been the majority-consensus definition of AGI. It seems to me that the Turing test has fundamentally always been about proving a negative: if something fails the Turing test, it's probably not AGI. For reference, the term AGI post-dates the Turing test by half a century. I also don't personally remember ever hearing the exact term "artificial general intelligence" prior to 2023 or 2024, or at least it wasn't mainstream the way it is today. If AGI had truly ever been defined by the Turing test, then Cleverbot should've been hailed as AGI when it passed the test in 2011. Even if we did all agree to call it that, we'd still need some other term for what we actually mean when we say "AGI" today. Cleverbot-era chatbots were cute toys, but they weren't capable of doing useful work of any kind.
- ninetyninenine 1y ago
  That’s not accurate. The Turing test was always intended as a benchmark for general intelligence. Turing’s 1950 paper explicitly proposed it as a way to operationalize the question “Can machines think?” not as a parlor trick about conversation but as a proxy for indistinguishability in intellectual behavior. The whole point of the imitation game was to sidestep metaphysical arguments and reduce intelligence to functional equivalence. If a machine could consistently hold its own in unrestricted dialogue, it would demonstrate the breadth, adaptability, and contextual understanding that characterize general intelligence. The term AGI may have come later, but the concept it represents traces directly back to Turing’s framing. When early AI researchers talked about “strong AI” or “thinking machines,” they were using the same conceptual lineage. The introduction of the acronym doesn’t rewrite that history, it just gave a modern label to an old idea. The Turing test was never meant to detect a “negative” but to give a concrete, falsifiable threshold for when positive claims of general intelligence might be justified. As for Cleverbot, it never truly passed the test in any rigorous or statistically sound sense. Those 2011 headlines were based on short exchanges with untrained judges and no control group. Passing a genuine Turing test requires sustained coherence, reasoning across domains, and the ability to handle novel input gracefully. Cleverbot couldn’t do any of that. It failed the spirit of the test even if it tricked a few people in the letter of it. By contrast, modern large language models can pass the Turing test in flying colors. They can maintain long, open-ended conversations, reason about complex subjects, translate, summarize, and solve problems across many domains. Most human judges would be unable to tell them apart from people in text conversation, not for a few sentences but for hours. Granted, one can often tell ChatGPT is an AI because of its long and overly descriptive replies, but that’s a stylistic artifact, not a limitation of intelligence. The remarkable thing is that you can simply instruct it to imitate casual human conversation, and it will do so convincingly, adjusting tone, rhythm, and vocabulary on command. In other words, the test can be passed both intentionally and effortlessly. The Turing test was never obsolete; we finally built systems that can truly meet it.

ninetyninenine 1y ago

>Generative AI is great, but it's not like you could just assign an arbitrary job to a present-day LLM, give it access to an expense account, and check in quarterly with reasonable expectations of useful progress. Has anyone tried this yet?

jdlshore 1y ago
There have been a few attempts at “building a business by following ChatGPT’s instructions.” All the ones I’m aware of failed, and were really just a scheme to grab eyeballs. There’s also a website showing a company’s attempt to use multiple LLMs to close its (real-world, but historical) accounting month-end books. They fail miserably with compounding errors.
buu700 1y ago
I'd be interested in seeing the results if so. Like someone with $500k to burn just handing it to an AI and telling it to found a startup, including incorporation, vibe coding and deploying a product, publishing a website, bringing on cofounders and/or advisors and/or employees as needed, fundraising, etc. My guess is finding agentic tooling that could even push the model hard enough to attempt that in earnest would be a problem, and the compute bills would add up quickly, but it would be a very interesting experiment. Probably with hilarious results, but seeing what the actual failure points are would be useful information. Maybe it would actually end up hiring and paying someone to do something insanely illegal.
famouswaffles 1y ago
Vending Bench ? https://andonlabs.com/evals/vending-bench https://andonlabs.com/evals/vending-bench