simonw

28 ms·

721.
by simonw 3mo ago
OpenAI are promising more details in the future: > We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.
722.
by simonw 3mo ago
Anthropic's Mythos moment earned them a two week period where they had the best available model and couldn't sell access to it... and by the time the US government allowed them to sell it again OpenAI had released GPT-5.6 and Fabl
723.
by simonw 3mo ago
> Which ones? The ones that shout about that one METR study, use it to imply that millions of developers are deceiving themselves in believing that tools they use every day are useful, then admit that they don't like and don't
724.
by simonw 3mo ago
Four years of personal exploration of the technology. (I had an argument on Lobste.rs the other day about why only believing aspects of software engineering that have survived peer review doesn't make sense to me: https://lo
725.
by simonw 3mo ago
I read this headline and thought "METR study"... and sure enough, a few paragraphs in: > Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they wher
726.
by simonw 3mo ago
The rationale in the ToS change and the rationale in the blog post don't match up. The ToS change grammar could do with tightening up as well.
727.
by simonw 3mo ago
When I said "misinterpretation" here I was talking about people assuming that the Ford story was a simple case of "Ford fired their employees because they thought AI could replace them, then had to hire them all back again&qu
728.
by simonw 3mo ago
Something doesn't quite smell right about this story. Here's a key paragraph from the Nikkei story that this Futurism story re-tells: > Companies disclose such future debt not in their balance sheets, but in annotations to thei
729.
by simonw 3mo ago
I look at history all the time . Why is the code like this? Usually git blame will show me either a commit message with extra context or a link to an issue thread. When was this bug first introduced? Git bisect can track it down almost aut
730.
by simonw 3mo ago
It's not that hard - there are plenty of those. They're mostly not very funny though.
731.
by simonw 3mo ago
Is the headline here ("OpenAI and Anthropic unite against open-weight AI risks to their bottom line") an editorialized version of the original "OpenAI and Anthropic unite against China's open models" or did Axios ch
732.
by simonw 3mo ago
I had real trouble deciding how to end this piece. There's no easy answer here. All of the options are bad in different ways! As a builder of software, I want access to the best possible tools to help me keep that software secure. As a
733.
by simonw 3mo ago
If the vulnerability is purely an open redirect that doesn't work for me. Clearly there was a hole in the software but I don't think open redirect is the likely initial problem. Hopefully we will find out for sure in a few days.
734.
by simonw 3mo ago
I had a look at that one and several other Nexus Repository issues that share the July 14th date. I don't think they're it, for two reasons: Firstly, they all credit named individuals who don't seem to have a relationship wit
735.
by simonw 3mo ago
The lesson OpenAI and Anthropic should have learned from the whole Fable export controls thing should have been "don't pull stunts with the US government". Turns out they can backfire.
736.
by simonw 3mo ago
> There's no reason that stronger network protections would need to be turned off for a research model that's not supposed to have internet access. They didn't turn off the network protections - it looks like they used the
737.
by simonw 3mo ago
I was not aware that the term "guardrails" has a universally agreed upon definition. I've certainly seen probabilistic classifiers referred to as guardrails many times by many different people. I usually don't use the te
738.
by simonw 3mo ago
How do you then use GET requests to create a malicious dataset package and publish that to Hugging Face in order to exploit their package building infrastructure?
739.
by simonw 3mo ago
I'm not crazy about this: > We acknowledge that many developers have started to embrace LLMs as a tool in their workflows. Some use it extensively and rarely code by hand, others delegate only specific tasks to it. We understand tha
740.
by simonw 3mo ago
I'd be very impressed by anyone who can produce that in Figma or Word in ten minutes.
741.
by simonw 3mo ago
Here's the language that makes me think they are taking this seriously: > We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will
742.
by simonw 3mo ago
That Alibaba one was a sandbox escape (I agree those are common) but what's new with the OpenAI story is an attack against another company. This wasn't a small attack either, Hugging Face published a security advisory for their us
743.
by simonw 3mo ago
It reminded me of a bunch of plotlines from Person of Interest as well.
744.
by simonw 3mo ago
I think formal verification has never looked better. The main reason formal verification has never really taken off is that it's difficult. LLMs are significantly more familiar with Lean and Rocq and TLA+ than most software engineers.
745.
by simonw 3mo ago
This fits: "OpenAI’s accidental cyberattack against Hugging Face is sci-fi that happened" Or "against HF"
746.
by simonw 3mo ago
It's bad OpSec by the research team. Their sandbox was not bulletproof and their monitoring was insufficient. It looks to me like their production models have a lot more monitoring than their research clusters.
747.
by simonw 3mo ago
I expect it must have been more than just an open redirect if it let the models then go on to execute a bunch of vulnerabilities against Hugging Face.
748.
by simonw 3mo ago
They almost certainly did, because that was the entire point of the exercise. They deliberately removed all of the safety filters from the model and set it loose on an extremely difficult set of cybersecurity challenges to see how well it w
749.
by simonw 3mo ago
It's the first report I've seen of a model both escaping a sandbox and then actively exploiting another company, when neither of those actions was intended.
750.
by simonw 3mo ago
You know this makes OpenAI look really bad , right? Hugging Face had to tell all of their users, many of them paying customers: > As a precaution, we recommend rotating any access tokens and reviewing recent activity on your account. If

More ›