3 ms·

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So m

by jimmar 4mo ago

People predict that in 50 years, no human will be driving a car, and people will be shocked that we let humans drive cars manually. Coding may be the same. So many vulnerabilities in code written by very competent programmers. Manually building large, complex systems without major bugs or security vulnerabilities seems to be a nearly impossible challenge.

morpheos137 4mo ago
there is little evidence for this prediction.
- cubefox 4mo ago
  The rapid progress in the last few years in this regard is pretty strong evidence in my opinion.
  - morpheos137 4mo ago
    https://news.ycombinator.com/item?id=48225426 https://news.ycombinator.com/item?id=48225426 there is a difference between a stunt and a viable product. diverless cars and agi are the fusion of Silicon Valley.
    - cubefox 4mo ago
      Unlike fusion, driverless cars are already a reality, there are just have a few kinks to work out. LLMs are also pretty close to AGI already. 50 years are more than enough to figure it out.
- sp527 4mo ago
  Oh there's plenty of evidence. Because a lot of these people have been committing to repos in public for over a decade. Wouldn't take much to show the world just how fallible human coders really are.
- scotty79 4mo ago
  What evidence would you expect to see if that was the case?
  - sfink 4mo ago
    Some numbers, however shaky, that AI-written code is secure. It could become that way, but thus far no evidence has been presented for it. The best we have right now is that you can spend $20 in tokens to write a patch and then $20K to find a vulnerability in it. First, that's not measuring the same thing. Second, it's not very impressive. 50 years is a long, long time, so I wouldn't bet against it. But I agree that we don't have evidence for it yet.
    - scotty79 4mo ago
      What are the numbers on how secure is human written code? We should have something to compare AI numbers to. It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.
      - sfink 4mo ago
        > What are the numbers on how secure is human written code? We should have something to compare AI numbers to. That's kind of what the article is about? Mythos is finding lots of security bugs in lots of human-written code. They can now compute some sort of baseline estimate of security bugs per N lines of human-written code or whatever. (Restricted to security bugs that the AI is currently capable of finding, but whatever.) Even before Mythos et al, we can look at historical security bug rates. We do have numbers for estimating the security of human written code. > It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor. Ok, but that's not what is being discussed in this subthread? The topic is whether or not we have data suggesting that AI-written code is or can be secure, and thus whether insecure human code is fated to replaced with secure AI code. I claim we do not have that data. Therefore, we don't have evidence to think that for the sake of security we should replace all human code with AI code, vs whether AI code is worse for security and so we should replace AI code with human code (that presumably has been vetted with AI, since we do have evidence for its effectiveness.) If I were to guess, I would probably think that today's AIs are trained solely on mountains of insecure human code and so will probably produce more of the same. Tomorrow's AIs will have the benefit of being trained on human and AI code that has had a large swathe of vulnerabilities purged from it, and so they'll have a much better chance at writing secure code, at least. It depends a lot on whether the failure modes of AI code generation lend themselves to exploitation as security vulnerabilities. (And whether they will continue to do so.)
cheesefck 4mo ago
Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails. This is the MoviePass era of language models
- Flere-Imsaho 4mo ago
  Actually I think with flying cars it's more of a problem with noise, regulation, risk, etc than a technological problem. Supersonic again is a problem with noise and cost rather than technological. Self driving is definitely a technological problem.
brightbeige 4mo ago
And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans. Here are two experimental exceptions: https://github.com/vercel-labs/zerolang https://github.com/vercel-labs/zerolang https://github.com/sbhooley/ainativelang https://github.com/sbhooley/ainativelang
- dmix 4mo ago
  Not just the languages but frontend/user interfaces as well. You can see the potential for the future when using Claude Design->Claude Code->Agents live testing in BrowserOS. It's all modeled on existing humans patterns of using Figma passing to devs then testing after the fact before starting the loop again, while a lot gets lost in translation in between the designs and the code. We'll like have some standard AI-focused UI libraries that are harnessed into a design gen system where an AI can pull all the real levers, while also developing a large training data set around it.
vb-8448 4mo ago
I just wonder how many of those 1451 acknowledged findings were introduced by LLMs ...
Oarch 4mo ago
I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.
- Gigachad 4mo ago
  In 50 years the world itself will be unrecognisable. The world could be a smouldering wreck by then.
3836293648 4mo ago
I hope this will never be the case. As long as we have personal vehicles they should be personally controlled. Self driving cars is such a waste of everyone's money. Cities should all have better public transport and out in the middle of nowhere you don't need self driving anyway. (And yes, personal cars should be entirely banned from cities)

by jimmar 4mo ago

morpheos137 4mo ago

there is little evidence for this prediction.

cubefox 4mo ago
The rapid progress in the last few years in this regard is pretty strong evidence in my opinion.
- morpheos137 4mo ago
  https://news.ycombinator.com/item?id=48225426 https://news.ycombinator.com/item?id=48225426 there is a difference between a stunt and a viable product. diverless cars and agi are the fusion of Silicon Valley.
  - cubefox 4mo ago
    Unlike fusion, driverless cars are already a reality, there are just have a few kinks to work out. LLMs are also pretty close to AGI already. 50 years are more than enough to figure it out.
sp527 4mo ago
Oh there's plenty of evidence. Because a lot of these people have been committing to repos in public for over a decade. Wouldn't take much to show the world just how fallible human coders really are.
scotty79 4mo ago
What evidence would you expect to see if that was the case?
- sfink 4mo ago
  Some numbers, however shaky, that AI-written code is secure. It could become that way, but thus far no evidence has been presented for it. The best we have right now is that you can spend $20 in tokens to write a patch and then $20K to find a vulnerability in it. First, that's not measuring the same thing. Second, it's not very impressive. 50 years is a long, long time, so I wouldn't bet against it. But I agree that we don't have evidence for it yet.
  - scotty79 4mo ago
    What are the numbers on how secure is human written code? We should have something to compare AI numbers to. It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor.
    - sfink 4mo ago
      > What are the numbers on how secure is human written code? We should have something to compare AI numbers to. That's kind of what the article is about? Mythos is finding lots of security bugs in lots of human-written code. They can now compute some sort of baseline estimate of security bugs per N lines of human-written code or whatever. (Restricted to security bugs that the AI is currently capable of finding, but whatever.) Even before Mythos et al, we can look at historical security bug rates. We do have numbers for estimating the security of human written code. > It seems more likely to me that you could spend $20 to find a vulnerability in a piece of software that costed you $20k in human labor. Ok, but that's not what is being discussed in this subthread? The topic is whether or not we have data suggesting that AI-written code is or can be secure, and thus whether insecure human code is fated to replaced with secure AI code. I claim we do not have that data. Therefore, we don't have evidence to think that for the sake of security we should replace all human code with AI code, vs whether AI code is worse for security and so we should replace AI code with human code (that presumably has been vetted with AI, since we do have evidence for its effectiveness.) If I were to guess, I would probably think that today's AIs are trained solely on mountains of insecure human code and so will probably produce more of the same. Tomorrow's AIs will have the benefit of being trained on human and AI code that has had a large swathe of vulnerabilities purged from it, and so they'll have a much better chance at writing secure code, at least. It depends a lot on whether the failure modes of AI code generation lend themselves to exploitation as security vulnerabilities. (And whether they will continue to do so.)

cheesefck 4mo ago

Musk has been predicting self driving cars next year for fifteen years. Fifty years ago, everyone was going to be flying supersonic all the time. Flying cars were just around the corner. Interplanetary travel. Everyone forgets the technology that fails. This is the MoviePass era of language models

Flere-Imsaho 4mo ago
Actually I think with flying cars it's more of a problem with noise, regulation, risk, etc than a technological problem. Supersonic again is a problem with noise and cost rather than technological. Self driving is definitely a technological problem.

brightbeige 4mo ago

And to consider AI agents are still mostly entirely limited to generating code in token-heavy programming languages designed to be written, tested and debugged by humans. Here are two experimental exceptions: https://github.com/vercel-labs/zerolang https://github.com/vercel-labs/zerolang https://github.com/sbhooley/ainativelang https://github.com/sbhooley/ainativelang

dmix 4mo ago
Not just the languages but frontend/user interfaces as well. You can see the potential for the future when using Claude Design->Claude Code->Agents live testing in BrowserOS. It's all modeled on existing humans patterns of using Figma passing to devs then testing after the fact before starting the loop again, while a lot gets lost in translation in between the designs and the code. We'll like have some standard AI-focused UI libraries that are harnessed into a design gen system where an AI can pull all the real levers, while also developing a large training data set around it.

vb-8448 4mo ago

I just wonder how many of those 1451 acknowledged findings were introduced by LLMs ...

Oarch 4mo ago

I reckon that in 50 years the very idea of code existing will be esoteric knowledge, a bit like binary. We simply won't care to think at that level of abstraction anymore.

Gigachad 4mo ago
In 50 years the world itself will be unrecognisable. The world could be a smouldering wreck by then.

3836293648 4mo ago

I hope this will never be the case. As long as we have personal vehicles they should be personally controlled. Self driving cars is such a waste of everyone's money. Cities should all have better public transport and out in the middle of nowhere you don't need self driving anyway. (And yes, personal cars should be entirely banned from cities)