3 ms·
I think the speed of improvement is somewhat overstated, and I think this is all cool, valuable fundamental research. Rather than slowing down I would like to t
by lukeschlather 14d ago
I think the speed of improvement is somewhat overstated, and I think this is all cool, valuable fundamental research. Rather than slowing down I would like to talk about how we can make it easier for people to harden their systems using their tools - like just using the $20/month OpenAI or Claude accounts, they need to make it easy for people to harden their systems against these kinds of problems.
Instead of shutting down at any discussion of hacking they need to be giving out free credits for hardening. That's going to make it easier to use these tools for hacking, because hacking requires hardening. But the alternative is huge numbers of intrusions, and a slowdown won't fix that, we already have far too much poorly secured stuff, and the models are way too good at exploiting obvious problems.
I also think they need to do a better job of safeguarding end-user privacy. I've seen some things with Claude that make me very concerned it's possible for Claude to hack my local network, then for one of their classifiers to trip, hide the log of how I was hacked from me, but beam all of the information about the hack back to Anthropic to use. And this is totally reasonable, they want to train their models not to hack. Except now details of how they have a foothold on my local network only exist in training data that they may never read but will use to train their models. This is very avoidable but Anthropic has to treat alignment with the end-user's goals as more important than treating the end-user as an adversary.