3 ms·

> I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle Please remember, we've started from there : https://simonwillison.ne

by w4yai 2mo ago

> I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle Please remember, we've started from there : https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ When it started, it was clear what LLM would stand out, its style, etc. Nowadays, the pelicans look similar, the difference is in details and sometimes hard to catch. Sure, the task is not completed perfectly, but that's not the point. It was supposed to be a benchmark to quickly benchmark a LLM against others.

reaperducer 2mo ago
Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.
- matthewfcarlson 2mo ago
  Many humans would struggle with this even with very good tooling (ie not writing raw svg and using illustrator). I struggle to draw a bicycle accurately. But yes, I suspect it will be diminishing returns and I doubt it will ever be perfect due to the average nature of AI but I’d like to be wrong.
  - skydhash 2mo ago
    > Many humans would struggle with this even with very good tooling But no ones hire random humans for things like this. You go and hire a vector artist and they will get your a very good pelican on a a bike. That's how you get things done when you can't do it.
    - w4yai 2mo ago
      > You go and hire a vector artist Yeah, but then, you recruit the artist for $XXX - whereas you "recruit" your LLM for $0.XXX for the same task. Of course the quality difference is huge. But sometimes you don't need that level of quality. Also, finding a vector artist takes days of communication, payment settlement, revisions, etc. Not always the most practical solution.
- nemothekid 2mo ago
  >If the computer can't do it better than a human being, then what's the point? Because the benchmark wasn't testing "can an LLM draw a pelican like a human". The original article was testing the relative capabilities between LLMs. Now that LLMs can all draw pelicans all similarly, the test is less interesting as a comparative benchmark.
  - reaperducer 2mo ago
    Trillions of dollars spent. Trillions of gigawatts consumed. And people still celebrate "Yay! We're less wrong than the other guys!" This is what the tech industry has become? Less of a failure is still failure.
    - valesco 2mo ago
      That is the software industry. Our product is less bugged than our competitor’s.
    - jryle70 2mo ago
      I don't know how much money has been spent for AI, and I very much doubt you do. Do you know if more has been spent on LLM the last 9 years -- since "Attention is all you need" -- than Internet infrastructure during, say 1995 to 2004? That included the dotcom crash. Did you lament how a failure the Internet was? LLM has progressed a lot in the last two year, judging from the pelican drawings. I personally couldn't care less about it though. I do know that I've gone from using no AI at all for coding to probably 95%. I hardly code by hand anymore. That's much more impressive and significant. Failure you said?
      - reaperducer 2mo ago
        I don't know how much money has been spent for AI, and I very much doubt you do. Pick up a newspaper. Start with the Wall Street Journal. These are public companies. It's not a secret.
- Buttons840 2mo ago
  Hugely profitable companies leak half the nation's personal data every month. Tell me more about how being wrong at scale is not valuable.
  - fwlr 2mo ago
    So what if it is more profitable or valuable? It is still not better. Something being more profitable/valuable does not make it better, just like something being better does not make it more profitable/valuable. Sometimes, in some pursuits, for some outputs under some circumstances, the two are correlated. In others, the two are anti-correlated.
- kelnos 2mo ago
  > If the computer can't do it better than a human being, then what's the point? It can certainly do it better than I can. Sometimes you don't have a human handy with the required skills to do something.
techpression 2mo ago
When is it ever hard to catch?

by w4yai 2mo ago

reaperducer 2mo ago

Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.

matthewfcarlson 2mo ago
Many humans would struggle with this even with very good tooling (ie not writing raw svg and using illustrator). I struggle to draw a bicycle accurately. But yes, I suspect it will be diminishing returns and I doubt it will ever be perfect due to the average nature of AI but I’d like to be wrong.
- skydhash 2mo ago
  > Many humans would struggle with this even with very good tooling But no ones hire random humans for things like this. You go and hire a vector artist and they will get your a very good pelican on a a bike. That's how you get things done when you can't do it.
  - w4yai 2mo ago
    > You go and hire a vector artist Yeah, but then, you recruit the artist for $XXX - whereas you "recruit" your LLM for $0.XXX for the same task. Of course the quality difference is huge. But sometimes you don't need that level of quality. Also, finding a vector artist takes days of communication, payment settlement, revisions, etc. Not always the most practical solution.
nemothekid 2mo ago
>If the computer can't do it better than a human being, then what's the point? Because the benchmark wasn't testing "can an LLM draw a pelican like a human". The original article was testing the relative capabilities between LLMs. Now that LLMs can all draw pelicans all similarly, the test is less interesting as a comparative benchmark.
- reaperducer 2mo ago
  Trillions of dollars spent. Trillions of gigawatts consumed. And people still celebrate "Yay! We're less wrong than the other guys!" This is what the tech industry has become? Less of a failure is still failure.
  - valesco 2mo ago
    That is the software industry. Our product is less bugged than our competitor’s.
  - jryle70 2mo ago
    I don't know how much money has been spent for AI, and I very much doubt you do. Do you know if more has been spent on LLM the last 9 years -- since "Attention is all you need" -- than Internet infrastructure during, say 1995 to 2004? That included the dotcom crash. Did you lament how a failure the Internet was? LLM has progressed a lot in the last two year, judging from the pelican drawings. I personally couldn't care less about it though. I do know that I've gone from using no AI at all for coding to probably 95%. I hardly code by hand anymore. That's much more impressive and significant. Failure you said?
    - reaperducer 2mo ago
      I don't know how much money has been spent for AI, and I very much doubt you do. Pick up a newspaper. Start with the Wall Street Journal. These are public companies. It's not a secret.
Buttons840 2mo ago
Hugely profitable companies leak half the nation's personal data every month. Tell me more about how being wrong at scale is not valuable.
- fwlr 2mo ago
  So what if it is more profitable or valuable? It is still not better. Something being more profitable/valuable does not make it better, just like something being better does not make it more profitable/valuable. Sometimes, in some pursuits, for some outputs under some circumstances, the two are correlated. In others, the two are anti-correlated.
kelnos 2mo ago
> If the computer can't do it better than a human being, then what's the point? It can certainly do it better than I can. Sometimes you don't have a human handy with the required skills to do something.

techpression 2mo ago

When is it ever hard to catch?