4 ms·

Generally speaking AI will make it easier to write more secure code. Tooling and automation help a lot with security and AI makes it easier to write good toolin

Generally speaking AI will make it easier to write more secure code. Tooling and automation help a lot with security and AI makes it easier to write good tooling. I would wager good money that in a few years the most security-focused companies will be relying heavily on AI somewhere in their software supply chain. So I don't think this policy is about security posture. No doubt human experts are reviewing the security-relevant patches anyway.

tho23i4234324 1y ago
I'd doubt this very much - LLMs hallucinate API calls and commit all sorts of subtle errors that you need to catch (esp. if you're on proprietary problems which it's not trained on). It's a good replacement for Google, but probably nothing close to what it's being hyped out to be by the capital allocators.
OtherShrezzing 1y ago
While LLMs are really good at generating content, one of their key weaknesses is their (relative) inability to detect _missing_ content. I'd argue that the most impactful software security bugs in the last couple of decades (Heartbleed etc) have been errors of omission, rather than errors of inclusion. This means LLMs are: 1) producing lots more code to be audited 2) poor at auditing that code for the most impactful class of bugs That feels like a dangerous combination.
guappa 1y ago
> Generally speaking AI will make it easier to write more secure code In my personal experience, not at all.
latexr 1y ago
> Generally speaking AI will make it easier to write more secure code. https://www.backslash.security/press-releases/backslash-security-reveals-in-new-research-that-gpt-4-1-other-popular-llms-generate-insecure-code-unless-explicitly-prompted https://www.backslash.security/press-releases/backslash-secu...
- duskwuff 1y ago
  Heh. Yup. And I'd be especially concerned about code written for QEMU, as it's an unusual type of application. There's lots of example code and other writings about security in web applications which a language model is likely to have encountered in its training; hypervisors are much less frequently discussed.

tho23i4234324 1y ago

I'd doubt this very much - LLMs hallucinate API calls and commit all sorts of subtle errors that you need to catch (esp. if you're on proprietary problems which it's not trained on). It's a good replacement for Google, but probably nothing close to what it's being hyped out to be by the capital allocators.

OtherShrezzing 1y ago

While LLMs are really good at generating content, one of their key weaknesses is their (relative) inability to detect _missing_ content. I'd argue that the most impactful software security bugs in the last couple of decades (Heartbleed etc) have been errors of omission, rather than errors of inclusion. This means LLMs are: 1) producing lots more code to be audited 2) poor at auditing that code for the most impactful class of bugs That feels like a dangerous combination.

guappa 1y ago

> Generally speaking AI will make it easier to write more secure code In my personal experience, not at all.

latexr 1y ago

> Generally speaking AI will make it easier to write more secure code. https://www.backslash.security/press-releases/backslash-security-reveals-in-new-research-that-gpt-4-1-other-popular-llms-generate-insecure-code-unless-explicitly-prompted https://www.backslash.security/press-releases/backslash-secu...

duskwuff 1y ago
Heh. Yup. And I'd be especially concerned about code written for QEMU, as it's an unusual type of application. There's lots of example code and other writings about security in web applications which a language model is likely to have encountered in its training; hypervisors are much less frequently discussed.