8 ms·

I love simonw content. It's like a day after I saw something interesting on X wrt AI, I can trust that there will be an article a day later by simonw where he t

I love simonw content. It's like a day after I saw something interesting on X wrt AI, I can trust that there will be an article a day later by simonw where he tried the things I would have wanted to try and explains them nicely. Thanks for that!

sureglymop 3y ago
Same sentiment. Its like he reads hacker news or similar sources and is generally on top of tech news and then has the curiosity to look more into it himself and do practical experiments. I try to do the same which is why I love this content.
jgalt212 3y ago
I feel differently. I think Simon's wasting his talents on an evolutionary dead end. If transformers and LLMs held the answer, self-driving cars would be getting better.
- simonw 3y ago
  As a daily user of LLMs for over a year, I'm confident that they're not a waste of my time. Even if development were to freeze, they didn't improve at all from this day onwards, and their many monumental flaws (prompt injection, hallucination, inability to reason etc) were never solved, I still think they'd be worth studying and using. I expect we could still spend years figuring out new capabilities in the models we already have access to today.
  - wunderwuzzi23 3y ago
    I'm very glad that you are focusing and helping educate the industry (both from pure tech but also security aspects). And agreed that as a daily user of LLMs myself, the potential (and practical use cases already) are enormous. E.g. Writing code and rapid prototyping will never be the same for me. By the way the last official answer I got from OpenAI regarding the markdown issue was that they thought it wouldn't be a problem because there will be mitigations. My assumption now is that they thought they could maybe fix indirect prompt injection, but it doesn't look like there is a solution. My current takeaway is that with LLMs, the assumption always needs to be that the LLM is capable of outputting any content, and the client needs to safely (and securely) handle whatever is returned. Btw. I show cased these kind of image based prompt injections in July (with Bard and Bing Chat, which presumably already used OpenAI's tech), and the example I created (the robot slipping on a banana) was shown during Blackhat in Las Vegas this year. Tweets: https://twitter.com/wunderwuzzi23/status/1681520761146834946 https://twitter.com/wunderwuzzi23/status/1681520761146834946
    - simonw 3y ago
      Just added your tweet there to my Twitter thread about this: https://twitter.com/simonw/status/1713344669181178088 https://twitter.com/simonw/status/1713344669181178088
- danShumway 3y ago
  Simon is bullish about LLMs, but his writing has been incredibly valuable regardless of which side of the LLM fence you're on. There are a limited number of authors describing injection attacks in accessible, detailed, accurate ways, and Simon has done a lot to raise awareness about these attacks. In addition, the fact that Simon is very bullish about LLMs makes his writing on LLM attacks more effective, because a decent chunk of people warning about LLM attacks are also skeptical about the long-term viability of the technology or think that it's over-hyped, so LLM fans tend to to dismiss them as being "anti-AI" and say that they're playing up the dangers. Nobody can make that claim about Simon. For whatever reason Simon's writing has consistently attracted a lot more attention than other researcher releases about LLM attacks; and what has he done with that popularity and outsized attention? He's raised awareness about the issue and pointed back to some of that research! I just don't think the criticism in this case is warranted, I think Simon's writing here is worthwhile regardless of how anyone feels about the potential of LLMs. Whether they're the future or not, these vulnerabilities exist today in products that are being rolled out today. That's worth taking about if LLMs aren't the future, but it's even more worth talking about if LLMs are the future.
- sebzim4500 3y ago
  What do LLMs have to do with self-driving cars? Also they are getting better, just slowly.
  - famouswaffles 3y ago
    Guess he's expecting something like this to happen faster? https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...
  - jgalt212 3y ago
    My analogy is that you cannot trust the outputs of an LLM or self-driving car.
- jug 3y ago
  Well, self-driving cars are already in production and use. They may not be amazing just yet but they have already passed the greatest hurdle there is.
  - ewngzen 3y ago
    self-driving cars already in production are not explainable, at least with llm in it, we know what the network is thinking, and thus are possible to figure out what went wrong

sureglymop 3y ago

Same sentiment. Its like he reads hacker news or similar sources and is generally on top of tech news and then has the curiosity to look more into it himself and do practical experiments. I try to do the same which is why I love this content.

jgalt212 3y ago

I feel differently. I think Simon's wasting his talents on an evolutionary dead end. If transformers and LLMs held the answer, self-driving cars would be getting better.

simonw 3y ago
As a daily user of LLMs for over a year, I'm confident that they're not a waste of my time. Even if development were to freeze, they didn't improve at all from this day onwards, and their many monumental flaws (prompt injection, hallucination, inability to reason etc) were never solved, I still think they'd be worth studying and using. I expect we could still spend years figuring out new capabilities in the models we already have access to today.
- wunderwuzzi23 3y ago
  I'm very glad that you are focusing and helping educate the industry (both from pure tech but also security aspects). And agreed that as a daily user of LLMs myself, the potential (and practical use cases already) are enormous. E.g. Writing code and rapid prototyping will never be the same for me. By the way the last official answer I got from OpenAI regarding the markdown issue was that they thought it wouldn't be a problem because there will be mitigations. My assumption now is that they thought they could maybe fix indirect prompt injection, but it doesn't look like there is a solution. My current takeaway is that with LLMs, the assumption always needs to be that the LLM is capable of outputting any content, and the client needs to safely (and securely) handle whatever is returned. Btw. I show cased these kind of image based prompt injections in July (with Bard and Bing Chat, which presumably already used OpenAI's tech), and the example I created (the robot slipping on a banana) was shown during Blackhat in Las Vegas this year. Tweets: https://twitter.com/wunderwuzzi23/status/1681520761146834946 https://twitter.com/wunderwuzzi23/status/1681520761146834946
  - simonw 3y ago
    Just added your tweet there to my Twitter thread about this: https://twitter.com/simonw/status/1713344669181178088 https://twitter.com/simonw/status/1713344669181178088
danShumway 3y ago
Simon is bullish about LLMs, but his writing has been incredibly valuable regardless of which side of the LLM fence you're on. There are a limited number of authors describing injection attacks in accessible, detailed, accurate ways, and Simon has done a lot to raise awareness about these attacks. In addition, the fact that Simon is very bullish about LLMs makes his writing on LLM attacks more effective, because a decent chunk of people warning about LLM attacks are also skeptical about the long-term viability of the technology or think that it's over-hyped, so LLM fans tend to to dismiss them as being "anti-AI" and say that they're playing up the dangers. Nobody can make that claim about Simon. For whatever reason Simon's writing has consistently attracted a lot more attention than other researcher releases about LLM attacks; and what has he done with that popularity and outsized attention? He's raised awareness about the issue and pointed back to some of that research! I just don't think the criticism in this case is warranted, I think Simon's writing here is worthwhile regardless of how anyone feels about the potential of LLMs. Whether they're the future or not, these vulnerabilities exist today in products that are being rolled out today. That's worth taking about if LLMs aren't the future, but it's even more worth talking about if LLMs are the future.
sebzim4500 3y ago
What do LLMs have to do with self-driving cars? Also they are getting better, just slowly.
- famouswaffles 3y ago
  Guess he's expecting something like this to happen faster? https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...
- jgalt212 3y ago
  My analogy is that you cannot trust the outputs of an LLM or self-driving car.
jug 3y ago
Well, self-driving cars are already in production and use. They may not be amazing just yet but they have already passed the greatest hurdle there is.
- ewngzen 3y ago
  self-driving cars already in production are not explainable, at least with llm in it, we know what the network is thinking, and thus are possible to figure out what went wrong