3 ms·

Must be nice to be in a position to sell both disease and cure.

supern0va 7mo ago
Yeah, I'd pretty pissed at my doctor for finding cancerous cells that probably wouldn't have been a problem for quite some time, either. Ignorance is bliss, security through obscurity, whatever.
- tredre3 7mo ago
  The doctor analogy is more like you're grateful that your doctor found cancerous cells before they became a problem, but at the same time his other business is selling cigarettes.
- ascorbic 7mo ago
  You may joke, but this is a genuine issue in certain screening tests. e.g. most cancerous cells found in PSA prostate screening are so slow growing that they never cause any symptoms during a person's lifetime, so the treatment is almost always worse than the disease. It's similar for some sorts of thyroid and breast cancer tests. This is why a lot of countries are heavily reducing these sort of tests
tptacek 7mo ago
That's exactly not what they're doing. They aren't creating operating system vulnerabilities. They're telling you about ones that already existed.
- conradkay 7mo ago
  Well, in a slightly indirect manner. Claude is writing a ton of code, and therefore creating a lot of security vulnerabilities.
  - tptacek 7mo ago
    That's not what's happening here. This announcement is about the velocity with which Claude finds vulnerabilities in already-existing software.
    - buzzerbetrayed 7mo ago
      Software already exists that has been written by Claude. They absolutely are selling the means to write software, and the means to securing the insecure software. At least for the time being. In the future Mythos will probably just make it possible to prompt good software from the start.
      - stale2002 7mo ago
        Ok. But mostly its entirely the old software, not the new software, that the bugs are being found in.
        pilgrim0 7mo ago
        Maybe because there’s no critical and widely used software written by LLMs so far? Which says a lot about LLMs are failing to even approach the level of capabilities you would expect from all the hype? The goal has always been, even before LLMs, to find something smarter than our smarter humans. So far the success at that is really minuscule. Humans are still the benchmark, all things considered. Now they’re saying LLMs are going to be better than our best vulnerability researchers in a few months (literally what an Anthropic researcher said in a conference). Ok, that might happen. But the funny part is that the LLMs will definitely be the ones writing most of these vulnerabilities. So, to hedge against LLMs you must use LLMs. And that is gonna cost you more.
        akerl_ 7mo ago
        So today, most of the vulnerabilities being found by these tools are in code written by humans. Your hypothesis is that down the road, most of the vulnerabilities will be in code written by LLMs. What seems more probable is that the same advances that LLMs are shipping to find vulnerabilities will end up baked into developer tooling. So you'll be writing code and using an LLM that knows how to write secure code.
  - dyauspitr 7mo ago
    If it’s very good at finding security vulnerabilities, I would assume that the code it generates is much more hardened than anything your average developer can put out.
- blazespin 7mo ago
  Dario is big on beating china, and no doubt he believes cyber security is how to do that. You can tell, but anthropic is sht at everything else. Nobody uses it for real research.
- josh-sematic 7mo ago
  Mythos aside, frontier LLMs can already be used to find exploits at faster pace than humans alone. Whether that knowledge gets used to patch them or exploit them is dependent on the user. Cybersecurity has always been an arms race and LLMs are rapidly becoming powerful arms. Whether they like it or not LLM providers are now important dealers in that arms race. I appreciate Anthropic trying to give “good guys” a leg up (if that is indeed their real main motivation which I do find credible but not certain). But it’s still a scary world we’re entering and I doubt the fierce competition will leave all labs acting benevolently.

Yeah, I'd pretty pissed at my doctor for finding cancerous cells that probably wouldn't have been a problem for quite some time, either. Ignorance is bliss, security through obscurity, whatever.

tredre3 7mo ago
The doctor analogy is more like you're grateful that your doctor found cancerous cells before they became a problem, but at the same time his other business is selling cigarettes.
ascorbic 7mo ago
You may joke, but this is a genuine issue in certain screening tests. e.g. most cancerous cells found in PSA prostate screening are so slow growing that they never cause any symptoms during a person's lifetime, so the treatment is almost always worse than the disease. It's similar for some sorts of thyroid and breast cancer tests. This is why a lot of countries are heavily reducing these sort of tests

tptacek 7mo ago

That's exactly not what they're doing. They aren't creating operating system vulnerabilities. They're telling you about ones that already existed.

conradkay 7mo ago
Well, in a slightly indirect manner. Claude is writing a ton of code, and therefore creating a lot of security vulnerabilities.
- tptacek 7mo ago
  That's not what's happening here. This announcement is about the velocity with which Claude finds vulnerabilities in already-existing software.
  - buzzerbetrayed 7mo ago
    Software already exists that has been written by Claude. They absolutely are selling the means to write software, and the means to securing the insecure software. At least for the time being. In the future Mythos will probably just make it possible to prompt good software from the start.
    - stale2002 7mo ago
      Ok. But mostly its entirely the old software, not the new software, that the bugs are being found in.
      - pilgrim0 7mo ago
        Maybe because there’s no critical and widely used software written by LLMs so far? Which says a lot about LLMs are failing to even approach the level of capabilities you would expect from all the hype? The goal has always been, even before LLMs, to find something smarter than our smarter humans. So far the success at that is really minuscule. Humans are still the benchmark, all things considered. Now they’re saying LLMs are going to be better than our best vulnerability researchers in a few months (literally what an Anthropic researcher said in a conference). Ok, that might happen. But the funny part is that the LLMs will definitely be the ones writing most of these vulnerabilities. So, to hedge against LLMs you must use LLMs. And that is gonna cost you more.
        akerl_ 7mo ago
        So today, most of the vulnerabilities being found by these tools are in code written by humans. Your hypothesis is that down the road, most of the vulnerabilities will be in code written by LLMs. What seems more probable is that the same advances that LLMs are shipping to find vulnerabilities will end up baked into developer tooling. So you'll be writing code and using an LLM that knows how to write secure code.
- dyauspitr 7mo ago
  If it’s very good at finding security vulnerabilities, I would assume that the code it generates is much more hardened than anything your average developer can put out.
blazespin 7mo ago
Dario is big on beating china, and no doubt he believes cyber security is how to do that. You can tell, but anthropic is sht at everything else. Nobody uses it for real research.
josh-sematic 7mo ago
Mythos aside, frontier LLMs can already be used to find exploits at faster pace than humans alone. Whether that knowledge gets used to patch them or exploit them is dependent on the user. Cybersecurity has always been an arms race and LLMs are rapidly becoming powerful arms. Whether they like it or not LLM providers are now important dealers in that arms race. I appreciate Anthropic trying to give “good guys” a leg up (if that is indeed their real main motivation which I do find credible but not certain). But it’s still a scary world we’re entering and I doubt the fierce competition will leave all labs acting benevolently.