4 ms·

I don’t think I’ve ever seen this guy say anything negative about an AI product, which makes me skeptical of his insights here.

asadotzler 9mo ago
He's a proponent, but that doesn't mean his analysis isn't useful. It's clear and mostly accurate and when he gets something wrong he makes it right. Does he do all that with rose tinted glasses, probably, but my experience reading him is that he's sharp, thoughtful, and entirely reasonable. Dismissing the opportunity to learn because the person offering you knowledge is enthusiastic about his area of expertise is probably shortsighted.
- Cornbilly 9mo ago
  I don't think they were being dismissive. They just said they were skeptical, which is generally a good thing. It's certainly better than the goofy hero worship I constantly see on HN.
- emp17344 9mo ago
  How is this Simon’s area of expertise? I know he’s a programming legend, but I’ve never heard anything to indicate he’s a machine learning expert. I’m not intending to be dismissive, just noticing a pattern and advocating a bit of skepticism.
  - wanderingstan 9mo ago
    This is more akin to a race car driver give a review of, for example, a new type of electric car. It doesn’t matter that the driver is not a domain expert in electric motors and regenerative braking; what matters is he knows how to operate these machines in their use case at the limits. Hearing a programming legend weigh in on the latest programming tool seems entirely completely reasonable.
    - deleted 9mo ago
      [deleted]
      - deleted 9mo ago
        [deleted]
  - simonw 9mo ago
    Being an expert in machine learning turns out to not be particularly relevant to being an expert in the applications of LLMs to real-world problems. I'm certainly not an expert in the former but I do think I have credibility in the latter.
webdevver 9mo ago
id imagine someone like Simon to pick his AI products carefully enough that he doesn't waste his time on duds.
simonw 9mo ago
Have you seen my writing on prompt injection (a term that I coined)? That's pretty negative! https://simonwillison.net/series/prompt-injection/ https://simonwillison.net/series/prompt-injection/ There's a whole section in the linked piece about how Cowork doesn't do enough here, including: > I do not think it is fair to tell regular non-programmer users to watch out for “suspicious actions that may indicate prompt injection”
- what 9mo ago
  I don’t think you coined “prompt injection”…
  - simonw 9mo ago
    https://simonwillison.net/2022/Sep/12/prompt-injection/ https://simonwillison.net/2022/Sep/12/prompt-injection/ > This isn’t just an interesting academic trick: it’s a form of security exploit. I propose that the obvious name for this should be prompt injection. I've written about it 142 times since then: https://simonwillison.net/tags/prompt-injection/ https://simonwillison.net/tags/prompt-injection/ I'm credited for coining the term on Wikipedia and in several academic papers. I don't claim to have discovered the vulnerability - I credited that to Riley Goodside, but we later learned it was independently discovered and first reported to OpenAI by Jonathan Cefalu of Preamble, see https://www.preamble.com/prompt-injection-a-critical-vulnerability-in-the-gpt-3-transformer-and-how-we-can-begin-to-solve-it https://www.preamble.com/prompt-injection-a-critical-vulnera...
mbesto 9mo ago
I would argue the EXACT opposite. His analysis is very constructive. The people you should be skeptical of are the random Xitter handles who post about a robotic phlebotomists and say "THE FUTUE IS ALREADY HERE".
- Cornbilly 9mo ago
  No. It’s also good to be skeptical of people that seem like they are operating in good faith. Example: The decade+ of people worshipping at Musk’s feet only for him to reveal himself as a malignant narcissist.
  - vacuity 9mo ago
    The deciding factor on whether to believe someone should not be whether they believe themselves, but whether they are worth believing. A murderer who acts in good faith is still a murderer.
mvdtnz 9mo ago
You're right to be skeptical. He makes a living as a hype merchant.
- simonw 9mo ago
  It's a pretty terrible way to make a living, to be honest. If I was in this for the money I'd trade my reputation as an independent voice for a six figure salary at an AI company.
pglevy 9mo ago
He literally brings up a concern he calls the "lethal trifecta" when it's even remotely relevant.

asadotzler 9mo ago

He's a proponent, but that doesn't mean his analysis isn't useful. It's clear and mostly accurate and when he gets something wrong he makes it right. Does he do all that with rose tinted glasses, probably, but my experience reading him is that he's sharp, thoughtful, and entirely reasonable. Dismissing the opportunity to learn because the person offering you knowledge is enthusiastic about his area of expertise is probably shortsighted.

Cornbilly 9mo ago
I don't think they were being dismissive. They just said they were skeptical, which is generally a good thing. It's certainly better than the goofy hero worship I constantly see on HN.
emp17344 9mo ago
How is this Simon’s area of expertise? I know he’s a programming legend, but I’ve never heard anything to indicate he’s a machine learning expert. I’m not intending to be dismissive, just noticing a pattern and advocating a bit of skepticism.
- wanderingstan 9mo ago
  This is more akin to a race car driver give a review of, for example, a new type of electric car. It doesn’t matter that the driver is not a domain expert in electric motors and regenerative braking; what matters is he knows how to operate these machines in their use case at the limits. Hearing a programming legend weigh in on the latest programming tool seems entirely completely reasonable.
  - deleted 9mo ago
    [deleted]
    - deleted 9mo ago
      [deleted]
- simonw 9mo ago
  Being an expert in machine learning turns out to not be particularly relevant to being an expert in the applications of LLMs to real-world problems. I'm certainly not an expert in the former but I do think I have credibility in the latter.

webdevver 9mo ago

id imagine someone like Simon to pick his AI products carefully enough that he doesn't waste his time on duds.

simonw 9mo ago

Have you seen my writing on prompt injection (a term that I coined)? That's pretty negative! https://simonwillison.net/series/prompt-injection/ https://simonwillison.net/series/prompt-injection/ There's a whole section in the linked piece about how Cowork doesn't do enough here, including: > I do not think it is fair to tell regular non-programmer users to watch out for “suspicious actions that may indicate prompt injection”

what 9mo ago
I don’t think you coined “prompt injection”…
- simonw 9mo ago
  https://simonwillison.net/2022/Sep/12/prompt-injection/ https://simonwillison.net/2022/Sep/12/prompt-injection/ > This isn’t just an interesting academic trick: it’s a form of security exploit. I propose that the obvious name for this should be prompt injection. I've written about it 142 times since then: https://simonwillison.net/tags/prompt-injection/ https://simonwillison.net/tags/prompt-injection/ I'm credited for coining the term on Wikipedia and in several academic papers. I don't claim to have discovered the vulnerability - I credited that to Riley Goodside, but we later learned it was independently discovered and first reported to OpenAI by Jonathan Cefalu of Preamble, see https://www.preamble.com/prompt-injection-a-critical-vulnerability-in-the-gpt-3-transformer-and-how-we-can-begin-to-solve-it https://www.preamble.com/prompt-injection-a-critical-vulnera...

mbesto 9mo ago

I would argue the EXACT opposite. His analysis is very constructive. The people you should be skeptical of are the random Xitter handles who post about a robotic phlebotomists and say "THE FUTUE IS ALREADY HERE".

Cornbilly 9mo ago
No. It’s also good to be skeptical of people that seem like they are operating in good faith. Example: The decade+ of people worshipping at Musk’s feet only for him to reveal himself as a malignant narcissist.
- vacuity 9mo ago
  The deciding factor on whether to believe someone should not be whether they believe themselves, but whether they are worth believing. A murderer who acts in good faith is still a murderer.

mvdtnz 9mo ago

You're right to be skeptical. He makes a living as a hype merchant.

simonw 9mo ago
It's a pretty terrible way to make a living, to be honest. If I was in this for the money I'd trade my reputation as an independent voice for a six figure salary at an AI company.

pglevy 9mo ago

He literally brings up a concern he calls the "lethal trifecta" when it's even remotely relevant.