3 ms·

I think there are more than enough people with limited domain understanding imagining that AI can do all sorts of things that it won't be able to do any time so

I think there are more than enough people with limited domain understanding imagining that AI can do all sorts of things that it won't be able to do any time soon, if ever. There's enough Yudkowskis fabulating the impossible and screaming about it. The bugs in the paper are largely low complexity issues that have been auto-exploitable by domain-specific programs in the past. I really wish everything Ai would get a much more sober and less hyperbolic treatment.

broknbottle 2y ago
How else do you propose that we pump the stock? If it’s not blockchain, NFTs, AI, GenAI, then we’ll need something. Stock won’t pump itself.
- 1oooqooq 2y ago
  depressing part is the current pump beats buying coca cola.
beardedwizard 2y ago
this. all these articles written about LLMs exploiting zero days completely miss the point. No researcher uses it this way, they use it to explore the semantic meaning of the code to discover and confirm the vulnerabilities. It is a more easily used SAST.
- mattgreenrocks 2y ago
  When established SMEs in VR or other domains start getting good results with LLMs, I’m happy to take a look. Until then, I remain suspicious of too-excited opinions on this topic because it is socially cheap and advantageous to grab onto hype waves, especially ones of this magnitude. It is also easy to underestimate the knowledge of SMEs if you don’t have the requisite domain knowledge yourself, and believe that LLMs get those details right.
dmd 2y ago
There's also a huge contingent of people loudly claiming that current AI can't do things that it demonstrably does every single day.
- brigadier132 2y ago
  Ok, give some examples
  - dmd 2y ago
    Like people who say that LLMs can't write code, they just are copying and pasting verbatim code they've seen before (or some people who even seem to believe that it's all Mechanical Turk!). Meanwhile millions are using it to write code every day; it's literally impossible to believe the above if you've every actually used it rather than just armchair-philosophized about it.
    - brigadier132 2y ago
      I was asking for real world examples. Your hypothetical of someone saying LLMs can't code can be interpreted as "LLMs can't code without significant oversight from humans". Which is factual. If I had a new hire that produced bugs and code that didn't work at the rate that LLMs produced bugs I would say they couldn't code.
      - dmd 2y ago
        No, I mean the large contingent of people who literally say that all the code, and all the images, produced by LLMs exist on the web already and that LLMs are just search engines. This is not a hypothetical.
        tdullien 2y ago
        Oh, I wasn't aware people seriously advocated that. I've heard people argue LLMs are approximate search engines and their results kernel-smoothed combinations of multiple terms in the index and the embedded prompt (and I can see some of that argument), but I hadn't seen anyone seriously argue they're pure info retrieval. That seems obviously wrong.
TeMPOraL 2y ago
> I think there are more than enough people with limited domain understanding imagining that AI can do all sorts of things that it won't be able to do any time soon, if ever. There's enough Yudkowskis fabulating the impossible and screaming about it. That is a bit rich given the jumps in model capabilities made in the past two years. Do you have any reason to believe the field is past its peak and slowing down?
- tdullien 2y ago
  A vast number if scenarios advocated by Yudkowski and adjacent people are anti-physics and rely on the world as we know it to operate under vastly different principles than we think right now. I think there's zero reason to think AIs will somehow find cheat codes to reality. Hence my "if ever". Do I believe we will eventually get auto-exploitation? Yes, I gave a lot of input to a friend of mine that (many years before the LLM craze) then had good results automating heap layout search. For certain classes of vulnerabilities and targets we will get auto-exploitation, and for some we already have them (even without AI or LLMs, unless you classify tree search as AI). Do I think we will get anything resembling the OP I replied to? No, that's pure (nonscience) fiction.
  - mewpmewp2 2y ago
    How would you conclude it is anti physics? What's some example of something being anti physics?
    - tdullien 2y ago
      "The nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer. "
      - TeMPOraL 2y ago
        That's what viruses do, roughly. Complete with falling from the sky en masse. Biology and nanotech are, if you excuse my modern Tamarian... Corporate, when two pictures are shown. Pam, at her desk. I don't know where that quote is from, but without surrounding context, the only thing I can nitpick is the "diamondoid" bit.

broknbottle 2y ago

How else do you propose that we pump the stock? If it’s not blockchain, NFTs, AI, GenAI, then we’ll need something. Stock won’t pump itself.

1oooqooq 2y ago
depressing part is the current pump beats buying coca cola.

beardedwizard 2y ago

this. all these articles written about LLMs exploiting zero days completely miss the point. No researcher uses it this way, they use it to explore the semantic meaning of the code to discover and confirm the vulnerabilities. It is a more easily used SAST.

mattgreenrocks 2y ago
When established SMEs in VR or other domains start getting good results with LLMs, I’m happy to take a look. Until then, I remain suspicious of too-excited opinions on this topic because it is socially cheap and advantageous to grab onto hype waves, especially ones of this magnitude. It is also easy to underestimate the knowledge of SMEs if you don’t have the requisite domain knowledge yourself, and believe that LLMs get those details right.

dmd 2y ago

There's also a huge contingent of people loudly claiming that current AI can't do things that it demonstrably does every single day.

brigadier132 2y ago
Ok, give some examples
- dmd 2y ago
  Like people who say that LLMs can't write code, they just are copying and pasting verbatim code they've seen before (or some people who even seem to believe that it's all Mechanical Turk!). Meanwhile millions are using it to write code every day; it's literally impossible to believe the above if you've every actually used it rather than just armchair-philosophized about it.
  - brigadier132 2y ago
    I was asking for real world examples. Your hypothetical of someone saying LLMs can't code can be interpreted as "LLMs can't code without significant oversight from humans". Which is factual. If I had a new hire that produced bugs and code that didn't work at the rate that LLMs produced bugs I would say they couldn't code.
    - dmd 2y ago
      No, I mean the large contingent of people who literally say that all the code, and all the images, produced by LLMs exist on the web already and that LLMs are just search engines. This is not a hypothetical.
      - tdullien 2y ago
        Oh, I wasn't aware people seriously advocated that. I've heard people argue LLMs are approximate search engines and their results kernel-smoothed combinations of multiple terms in the index and the embedded prompt (and I can see some of that argument), but I hadn't seen anyone seriously argue they're pure info retrieval. That seems obviously wrong.

TeMPOraL 2y ago

> I think there are more than enough people with limited domain understanding imagining that AI can do all sorts of things that it won't be able to do any time soon, if ever. There's enough Yudkowskis fabulating the impossible and screaming about it. That is a bit rich given the jumps in model capabilities made in the past two years. Do you have any reason to believe the field is past its peak and slowing down?

tdullien 2y ago
A vast number if scenarios advocated by Yudkowski and adjacent people are anti-physics and rely on the world as we know it to operate under vastly different principles than we think right now. I think there's zero reason to think AIs will somehow find cheat codes to reality. Hence my "if ever". Do I believe we will eventually get auto-exploitation? Yes, I gave a lot of input to a friend of mine that (many years before the LLM craze) then had good results automating heap layout search. For certain classes of vulnerabilities and targets we will get auto-exploitation, and for some we already have them (even without AI or LLMs, unless you classify tree search as AI). Do I think we will get anything resembling the OP I replied to? No, that's pure (nonscience) fiction.
- mewpmewp2 2y ago
  How would you conclude it is anti physics? What's some example of something being anti physics?
  - tdullien 2y ago
    "The nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer. "
    - TeMPOraL 2y ago
      That's what viruses do, roughly. Complete with falling from the sky en masse. Biology and nanotech are, if you excuse my modern Tamarian... Corporate, when two pictures are shown. Pam, at her desk. I don't know where that quote is from, but without surrounding context, the only thing I can nitpick is the "diamondoid" bit.