4 ms·
I'm curious, you say "super interesting engineering" but then they say "it will often just tell you, even though we’ve asked it not to" and to me that seems lik
by embedding-shape 1mo ago
I'm curious, you say "super interesting engineering" but then they say "it will often just tell you, even though we’ve asked it not to" and to me that seems like extremely shit engineering.
Where are the interesting engineering parts at? Seems to be an interesting idea and perhaps design, but to call the implementation/engineering itself bad seems to be an understatement.
- fc417fc802 1mo agoThe security and the overall engineering were entirely separate items in that comment I think. It was explicitly called out that this is a security problem that you'd really only see treated in this manner in the LLM space. For what it's worth it's effectively unsolvable (AFAIU) short of realizing AGI with an amicable alignment.
- embedding-shape 1mo agoWhat do you mean unsolvable? Don't give the LLM access to stuff it shouldn't, this is like Access Control 101, not sure how anyone can claim that particular problem is unsolvable?
- fc417fc802 1mo agoJust don't give the hammer access to the nails they said ...
- robrorcroptrer 1mo agoThe LLM and agent harness at OpenAI accessed the internals of Huggingface.
- embedding-shape 1mo agoWhich again, pretty huge fuckup on OpenAI's side to run that sort of security testing on 3rd party hardware, and not on a airgapped machine, kind of amateur hour to be honest. Again, same principle, don't give it access to stuff you don't want it to access.
- bbor 1mo agoGood engineering means optimizing the things you care about, at the cost of things you don't. In this case, secrecy doesn't matter kinda inherently. It's like criticizing Reddit for not handling SSNs well; the intended design is 'just don't do that, cause you'd never have a reason to, anyway'
- rendaw 1mo agoIf secrecy doesn't matter, why did they tell the agent not to divulge stuff in the first place?