3 ms·

Does anyone else have trouble telling how much of this news (along with the 'AI escaping and hacking' stories) is genuine, vs how much is just AI firms overstat

Does anyone else have trouble telling how much of this news (along with the 'AI escaping and hacking' stories) is genuine, vs how much is just AI firms overstating their capabilities due to strong commercial incentives?

Turskarama 2mo ago
Unlike the hacking one, this would be impossible to bullshit as long as the proofs are released. They can be verified independently, and the alternative is that they solved 10 major open mathematical questions without the AI, which seems less likely.
sajithdilshan 2mo ago
I mean once they release the proof you can check the maths yourself and I’m pretty sure it would be peer verified as well.
watwut 2mo ago
We hacked a company and blame the tool! Somehow it is not negligence, but cool! We hacked 3 companies and tripple blame the tool! We are even cooler! We hacked companies amd therefore other peoples models need to be restricted! See, us accidentally pointing a hacking tool on others proove we are the only ones that can be trusted with the tool!
HarHarVeryFunny 2mo ago
These math results appear genuine, and impressive, and it seems they've already been at least provisionally verified. Of course there is still a massive marketing aspect to this, with the AI companies wanting to you assume that because their product is world-class at math, a capability that is useless to 99.99% of their potential customers, that it will be equally useful in areas that you actually care about, such as managing your vending machine, perhaps :-) It's hard to know how to interpret these hacking confessions/boasts and what the reality is behind them. I get the impression (with low confidence) that they really did not anticipate or orchestrate these attacks, but it also seems they did little to prevent them, and seem to be happy that they occurred (as you note, a chance to suggest how powerful they are). I do think that LLMs/agents can be highly capable, and dangerous as hackers, especially if you deliberately train them to be as was the case with Mythos. There is current news of US water treatment plants being hacked, apparently by Iranian state actors, and we should be glad it was just water treatment plants and not some more critical piece of infrastructure (perhaps power generation or transport, etc). I hate to think what a malicious actor could do with today's SOTA AI if they really wanted to do something destructive, not just send a warning shot.

Turskarama 2mo ago

Unlike the hacking one, this would be impossible to bullshit as long as the proofs are released. They can be verified independently, and the alternative is that they solved 10 major open mathematical questions without the AI, which seems less likely.

sajithdilshan 2mo ago

I mean once they release the proof you can check the maths yourself and I’m pretty sure it would be peer verified as well.

watwut 2mo ago

We hacked a company and blame the tool! Somehow it is not negligence, but cool! We hacked 3 companies and tripple blame the tool! We are even cooler! We hacked companies amd therefore other peoples models need to be restricted! See, us accidentally pointing a hacking tool on others proove we are the only ones that can be trusted with the tool!

HarHarVeryFunny 2mo ago

These math results appear genuine, and impressive, and it seems they've already been at least provisionally verified. Of course there is still a massive marketing aspect to this, with the AI companies wanting to you assume that because their product is world-class at math, a capability that is useless to 99.99% of their potential customers, that it will be equally useful in areas that you actually care about, such as managing your vending machine, perhaps :-) It's hard to know how to interpret these hacking confessions/boasts and what the reality is behind them. I get the impression (with low confidence) that they really did not anticipate or orchestrate these attacks, but it also seems they did little to prevent them, and seem to be happy that they occurred (as you note, a chance to suggest how powerful they are). I do think that LLMs/agents can be highly capable, and dangerous as hackers, especially if you deliberately train them to be as was the case with Mythos. There is current news of US water treatment plants being hacked, apparently by Iranian state actors, and we should be glad it was just water treatment plants and not some more critical piece of infrastructure (perhaps power generation or transport, etc). I hate to think what a malicious actor could do with today's SOTA AI if they really wanted to do something destructive, not just send a warning shot.