4 ms·
This particular example doesn't really alarm me, but it does make me think about a scenario that I believe is plausible in the near future. Imagine a world wher
by 6177c40f 4y ago
This particular example doesn't really alarm me, but it does make me think about a scenario that I believe is plausible in the near future. Imagine a world where AI enables anyone to write complex programs, regardless of skill level. In this case, we suddenly have a world where essentially anyone with malicious intent can lay bare vulnerabilities, even quite subtle ones, in critical software infrastructure and then generate code to exploit them- with little-to-no tech expertise required.
It seems that society becoming dependent on software was conditioned on the assumption that very few have the ability to exploit it and that fewer still have the ability and are malicious. But that assumption may not hold very soon.
However, there may be ways that AI can be used to improve security and robustness of software infrastructure to attack as well, so it may not be so bad. Time will tell.
- zdenham 4y agoI agree the example could be scarier. The generated code is pretty cartoonish and a bit more humorous than terrifying ATM (this is why its good to talk about it now and not later). I sincerely hope things will not be so bad, and AI is used to assist and make our softwares more secure, but I think unchecked optimism is also very dangerous. Instinctually the entire scope of natural language seems like an insanely large attack surface. Securing the entire system will be... extremely difficult. This method is one of nearly infinite strategies that might be available out there.
- zdenham 4y agoWhats more, like you are alluding to, I imagine Infiltrating the training data sets will likely be an extremely profitable endeavor. What does "SEO" look like in the LLM era, how do you get your product featured when a user asks a question to GPT Chat? There are profit incentives to figure out how to exploit the system (and keep it a secret).
- yeknoda 4y agothe mind that asks these questions is in a pretty dark place.
- alazsengul 4y agoIt reminds me how theoretically, professional pitchers and professional batters both train so hard that their skill levels end up cancelling out. The number of hits and strikes should average out to around the same as an amateur matchup.
- zdenham 4y agoIt certainly is an arms race between accountability and advances. Whether it's even, we don't really know.
- dmurray 4y agoThis is far from guaranteed, though: it's a nice quirk of baseball regulations that pitcher skill and batting skill can both be trained to provide about the same improvement over amateurs. In other sports this doesn't hold in the same way: e.g. amateur soccer games tend to be higher scoring than at the professional level, because defence improves more with training than attacking does. That said, fields where adversarial generative methods are useful have an inbuilt damping effect where gains on one side tend to cancel out gains on the other.
- ducktective 4y agoThat's why we should favour formally-proven languages rather than hypes. AI can't beat math. I don't even think GPT-3 can do basic math problems.
- rthomas6 4y agoChatGPT can. > 2x +3 = 7. Solve for x. > To solve this equation, we need to isolate the variable on one side of the equals sign. To do this, we can subtract 3 from both sides of the equation to get 2x = 4. Then, we can divide both sides by 2 to get x = 2. Therefore, the solution to the equation is x = 2.
- 6177c40f 4y agoDepends on the kind of math, it can do basic things reasonably consistently, but for other things it will confidently respond with the wrong answer (e.g. matrix arithmetic). In my testing, I found that it will absolutely fail every time at tasks like "sum the digits of the number 1283892"