3 ms·

Curious if people think LLMs will lead to more secure or less secure software in five years.

by crummy 5mo ago

int32_64 5mo ago
Both. The skilled will use them to find problems, the unskilled will use them to slopcode insecure software the skilled will have to fix.
deleted 5mo ago
[deleted]
stavros 5mo ago
That depends on which side has more money.
bawolff 5mo ago
One of the biggest issues in security historically imo is vendors who think, well nobody will ever find this bug so we can deprioritize fixing it. LLMs will prevent vendors lying to themselves which will lead to more secure software.
UltraSane 5mo ago
In 5 years attackers have an advantage but in the long run I think more secure if developers use LLMs on software to find and fix all of the worse remotely exploitable bugs before release. LLMs are going to force devs to be much more security conscious.
- canucker2016 5mo ago
  I think it'll be a war of who has the better LLMs-as-security-scanner. Ideally, you'd do a comprehensive all-source-code scan, (and the LLM-scanner finds everything during those scans), and fix all the reported defects. Afterwards, any dev that commits code will run the LLM-scanner on the modified code (and affected areas) and fix any reported defects. So the black-hat hacker would be shut out unless they get access to an LLM-scanner with better analysis than what the target project is using. Major LLM-scanners could give priority access for new versions of LLM-scanners to major projects to find any defects in the current source code before any other party could use the reported defects against the project or their users. So black-hat hackers would be left with developing their own LLM-scanner better/more efficient than existing major LLM-scanners. Given enough incentive, they might develop such a tool. Look at the market for zero-day vulnerabilities for smartphones, esp iPhones.
data-ottawa 5mo ago
I’m just happy we’re talking about security. That will make software safer alone.
mc3301 5mo ago
Kinda like home-improvement stores, power tools, easily available hardware and youtube tutorials led to both incredibly amazing and durable furniture, as well as janky, ugly and even dangerous furniture. More tools for more people equals more stuff being made on a wider range.
FeepingCreature 5mo ago
More secure software, but in the same way that the population is net healthier after a plague.
jillesvangurp 5mo ago
It will probably wipe out a few categories of issues, which is probably a good thing. And those things that still are still insecure can also be translated to some other language. Translating things to Rust manually was already a thing before LLMs came into the picture. Now with LLMs that's only going to get easier and faster. The long term value is going to come from getting on top of the mountain of technical debt in the form of existing C/C++ code bases that is responsible for the vast majority of memory exploits, buffer overflows, and other issues that despite decades of attention still are being found across major code bases on a regular basis. Mozilla finding these issues comes on the back of a quarter century of some very competent engineers trying to do the right thing and using all the tools at their disposal to prevent these issues from happening. I have a lot of respect for that team and the contributions it has made over the years to improve tools, testing/verification practices, etc. The issue is not their effort or competence. The job of taking an existing system that is well covered in test, well documented/specified, etc. and producing a new one that can function as a drop in replacement is now something that can be considered. A few years ago that would have translated into absolutely massive project cost and risk. Now it's something you can kick off on a Friday afternoon. Worst case it doesn't work, best case you end up with a much better implementation. It's still early days. There are still a lot of quality issues with LLM generated code. But the success/fail rate will probably improve over time.
vga1 5mo ago
More secure, at least in the cases where the tools are properly applied. But it also represents more easily available opportunities for blackhats to abuse against the projects where these tools were not being applied.
2ndorderthought 5mo ago
Less secure because of all the ways attacks can scale out and hackers can contribute vulnerabilities to active projects.

Curious if people think LLMs will lead to more secure or less secure software in five years.

by crummy 5mo ago

Curious if people think LLMs will lead to more secure or less secure software in five years.

Both. The skilled will use them to find problems, the unskilled will use them to slopcode insecure software the skilled will have to fix.

deleted 5mo ago

[deleted]

stavros 5mo ago

That depends on which side has more money.

bawolff 5mo ago

One of the biggest issues in security historically imo is vendors who think, well nobody will ever find this bug so we can deprioritize fixing it. LLMs will prevent vendors lying to themselves which will lead to more secure software.

UltraSane 5mo ago

In 5 years attackers have an advantage but in the long run I think more secure if developers use LLMs on software to find and fix all of the worse remotely exploitable bugs before release. LLMs are going to force devs to be much more security conscious.

canucker2016 5mo ago
I think it'll be a war of who has the better LLMs-as-security-scanner. Ideally, you'd do a comprehensive all-source-code scan, (and the LLM-scanner finds everything during those scans), and fix all the reported defects. Afterwards, any dev that commits code will run the LLM-scanner on the modified code (and affected areas) and fix any reported defects. So the black-hat hacker would be shut out unless they get access to an LLM-scanner with better analysis than what the target project is using. Major LLM-scanners could give priority access for new versions of LLM-scanners to major projects to find any defects in the current source code before any other party could use the reported defects against the project or their users. So black-hat hackers would be left with developing their own LLM-scanner better/more efficient than existing major LLM-scanners. Given enough incentive, they might develop such a tool. Look at the market for zero-day vulnerabilities for smartphones, esp iPhones.

data-ottawa 5mo ago

I’m just happy we’re talking about security. That will make software safer alone.

mc3301 5mo ago

Kinda like home-improvement stores, power tools, easily available hardware and youtube tutorials led to both incredibly amazing and durable furniture, as well as janky, ugly and even dangerous furniture. More tools for more people equals more stuff being made on a wider range.

FeepingCreature 5mo ago

More secure software, but in the same way that the population is net healthier after a plague.

jillesvangurp 5mo ago

It will probably wipe out a few categories of issues, which is probably a good thing. And those things that still are still insecure can also be translated to some other language. Translating things to Rust manually was already a thing before LLMs came into the picture. Now with LLMs that's only going to get easier and faster. The long term value is going to come from getting on top of the mountain of technical debt in the form of existing C/C++ code bases that is responsible for the vast majority of memory exploits, buffer overflows, and other issues that despite decades of attention still are being found across major code bases on a regular basis. Mozilla finding these issues comes on the back of a quarter century of some very competent engineers trying to do the right thing and using all the tools at their disposal to prevent these issues from happening. I have a lot of respect for that team and the contributions it has made over the years to improve tools, testing/verification practices, etc. The issue is not their effort or competence. The job of taking an existing system that is well covered in test, well documented/specified, etc. and producing a new one that can function as a drop in replacement is now something that can be considered. A few years ago that would have translated into absolutely massive project cost and risk. Now it's something you can kick off on a Friday afternoon. Worst case it doesn't work, best case you end up with a much better implementation. It's still early days. There are still a lot of quality issues with LLM generated code. But the success/fail rate will probably improve over time.

vga1 5mo ago

More secure, at least in the cases where the tools are properly applied. But it also represents more easily available opportunities for blackhats to abuse against the projects where these tools were not being applied.

2ndorderthought 5mo ago

Less secure because of all the ways attacks can scale out and hackers can contribute vulnerabilities to active projects.