4 ms·

It's not really passing the Turing Test until it outsells Harry Potter.

eru 2y ago
Well, strictly speaking outselling the Harry Potter would fail the Turing test: the Turing test is about passing for human (in an adversarial setting), not to surpass humans. Of course, this is just some pedantry. I for one love that AI is progressing so quickly, that we _can_ move the goalposts like this.
jychang 2y ago
To be fair, pacing as a big flaw of LLMs has been a constant complaint from writers for a long time. There were popular writeups about this from the Deepseek-R1 era: https://www.tumblr.com/nostalgebraist/778041178124926976/hydrogen-jukeboxes https://www.tumblr.com/nostalgebraist/778041178124926976/hyd...
- newswasboring 2y ago
  This was written on march 15. Deepseek came out in January. "Era" is not a language I would use for something that happened few days ago
dragonwriter 2y ago
> It's not really passing the Turing Test until it outsells Harry Potter. Most human-written books don't do that, so that seems to be a ceiteria for a very different test that a Turing test.
- mirekrusin 2y ago
  The joke is that the goalpost is constantly moving.
  - TeMPOraL 2y ago
    This subgoal post can't move much further after it passes "outsells the Bible" mark.
    - mirekrusin 2y ago
      You may be right, on the other hand it always feels like the next goalpost is the final one. I'm pretty sure if something like this happens some dude will show up from nowhere and claim that it's just parroting what other, real people have written, just blended it together and randomly spitted it out – "real AI would come up with original ideas like cure for cancer" he'll say. After some form of that comes another dude will show up and say that this "alphafold while-loop" is not real AI because he just went for lunch and there was a guy flipping burgers – and that "AI" can't do it so it's shit. https://areweagiyet.com https://areweagiyet.com should plot those future points as well with all those funky goals like "if Einstein had access to the Internet, Wolfram etc. he could came up with it anyway so not better than humans per se", or "had to be prompted and guided by human to find this answer so didn't do it by itself really" etc.
    - zimbatm 2y ago
      Why would the book be worth buying tough. If AI can generate a fresh new one just for you?
      - TeMPOraL 2y ago
        I don't know. It's a question relevant to all generative AI applications in entertainment - whether books, art, music, film or videogames. To the extent the value of these works is mostly in being social objects (i.e. shared experience to talk about with other people), being able to generate clones and personalized variants freely via GenAI destroys that value.
- ZiiS 2y ago
  Both books that have outsold the Harry Potter series claim divine authorship, not purely human. I am prepared to bet quite a lot that the next isn't human-written, either.
ninetyninenine 2y ago
the goal posts will be moved again. Tons of people clamoring the book is stupid and vapid and only idiots bought the book. When ai starts taking over jobs which it already has you’ll get tons of idiots claiming the same thing.
tummler 2y ago
Let me toss a grenade in here. What if we didn’t measure success by sales, but impact to the industry (or society), or value to peoples’ lives? Zooming out to AI broadly: what if we didn’t measure intelligence by (game-able, arguably meaningless) benchmarks, but real world use cases, adaptability, etc?
- Workaccount2 2y ago
  This is difficult to do because one of the juiciest parts of AI is being able to take credit for it's work.
- szatkus 2y ago
  I recently watched some Claude Plays Pokemon and believe it's better measure than all those AI benchmarks. The game could be beaten by a 8yo which obviously doesn't have all that knowledge that even small local LLMs posess, but has actual intelligence and could figure out the game within < 100h. So far Claude can't even get past the first half and I doubt any other AI could get much further.
  - harrison_clarke 2y ago
    the true measure of AI: does it have fun playing pokemon? did it make friends along the way?
  - solardev 2y ago
    Now I want to watch Claude play Pokemon Go, hitching a ride on self-driving cars to random destinations and then trying to autonomously interpret a live video feed to spin the ball at the right pixels... 2026 news feed: Anthropic cited as AI agents simultaneously block traffic across 42 major cities while trying to capture a not-even-that-rare pokemon
- etruong42 2y ago
  We humans love quantifiability. Since you used the word "measure", do you believe the measurement you're aspiring for is quantifiable? I currently assert that it's not, but I would also say that trying to follow your suggestion is better than our current approach of measuring everything by money.
silveraxe93 2y ago
From Gary Marcus' (notable AI skeptic) predictions of what AI won't do in 2027: > With little or no human involvement, write Pulitzer-caliber books, fiction and non-fiction. So, yeah. I know you made a joke, but you have the same issue as the Onion I guess.

eru 2y ago

Well, strictly speaking outselling the Harry Potter would fail the Turing test: the Turing test is about passing for human (in an adversarial setting), not to surpass humans. Of course, this is just some pedantry. I for one love that AI is progressing so quickly, that we _can_ move the goalposts like this.

jychang 2y ago

To be fair, pacing as a big flaw of LLMs has been a constant complaint from writers for a long time. There were popular writeups about this from the Deepseek-R1 era: https://www.tumblr.com/nostalgebraist/778041178124926976/hydrogen-jukeboxes https://www.tumblr.com/nostalgebraist/778041178124926976/hyd...

newswasboring 2y ago
This was written on march 15. Deepseek came out in January. "Era" is not a language I would use for something that happened few days ago

dragonwriter 2y ago

> It's not really passing the Turing Test until it outsells Harry Potter. Most human-written books don't do that, so that seems to be a ceiteria for a very different test that a Turing test.

mirekrusin 2y ago
The joke is that the goalpost is constantly moving.
- TeMPOraL 2y ago
  This subgoal post can't move much further after it passes "outsells the Bible" mark.
  - mirekrusin 2y ago
    You may be right, on the other hand it always feels like the next goalpost is the final one. I'm pretty sure if something like this happens some dude will show up from nowhere and claim that it's just parroting what other, real people have written, just blended it together and randomly spitted it out – "real AI would come up with original ideas like cure for cancer" he'll say. After some form of that comes another dude will show up and say that this "alphafold while-loop" is not real AI because he just went for lunch and there was a guy flipping burgers – and that "AI" can't do it so it's shit. https://areweagiyet.com https://areweagiyet.com should plot those future points as well with all those funky goals like "if Einstein had access to the Internet, Wolfram etc. he could came up with it anyway so not better than humans per se", or "had to be prompted and guided by human to find this answer so didn't do it by itself really" etc.
  - zimbatm 2y ago
    Why would the book be worth buying tough. If AI can generate a fresh new one just for you?
    - TeMPOraL 2y ago
      I don't know. It's a question relevant to all generative AI applications in entertainment - whether books, art, music, film or videogames. To the extent the value of these works is mostly in being social objects (i.e. shared experience to talk about with other people), being able to generate clones and personalized variants freely via GenAI destroys that value.
ZiiS 2y ago
Both books that have outsold the Harry Potter series claim divine authorship, not purely human. I am prepared to bet quite a lot that the next isn't human-written, either.

ninetyninenine 2y ago

the goal posts will be moved again. Tons of people clamoring the book is stupid and vapid and only idiots bought the book. When ai starts taking over jobs which it already has you’ll get tons of idiots claiming the same thing.

tummler 2y ago

Let me toss a grenade in here. What if we didn’t measure success by sales, but impact to the industry (or society), or value to peoples’ lives? Zooming out to AI broadly: what if we didn’t measure intelligence by (game-able, arguably meaningless) benchmarks, but real world use cases, adaptability, etc?

Workaccount2 2y ago
This is difficult to do because one of the juiciest parts of AI is being able to take credit for it's work.
szatkus 2y ago
I recently watched some Claude Plays Pokemon and believe it's better measure than all those AI benchmarks. The game could be beaten by a 8yo which obviously doesn't have all that knowledge that even small local LLMs posess, but has actual intelligence and could figure out the game within < 100h. So far Claude can't even get past the first half and I doubt any other AI could get much further.
- harrison_clarke 2y ago
  the true measure of AI: does it have fun playing pokemon? did it make friends along the way?
- solardev 2y ago
  Now I want to watch Claude play Pokemon Go, hitching a ride on self-driving cars to random destinations and then trying to autonomously interpret a live video feed to spin the ball at the right pixels... 2026 news feed: Anthropic cited as AI agents simultaneously block traffic across 42 major cities while trying to capture a not-even-that-rare pokemon
etruong42 2y ago
We humans love quantifiability. Since you used the word "measure", do you believe the measurement you're aspiring for is quantifiable? I currently assert that it's not, but I would also say that trying to follow your suggestion is better than our current approach of measuring everything by money.

silveraxe93 2y ago

From Gary Marcus' (notable AI skeptic) predictions of what AI won't do in 2027: > With little or no human involvement, write Pulitzer-caliber books, fiction and non-fiction. So, yeah. I know you made a joke, but you have the same issue as the Onion I guess.