4 ms·
Wow, the model weights themselves on that have to be extremely dangerous in the wrong hands.
by CGamesPlay 3y ago
Wow, the model weights themselves on that have to be extremely dangerous in the wrong hands.
- vlovich123 3y agoNo kidding. Run the model as a GAN and recover a lot of sensitive info.
- yanovskishai 3y agoor at least look like sensitive info .. :-)
- vlovich123 3y agoCross correlate with other methods and sources. Intelligence agencies already deal with shaky intel and sift things out. This just gives essentially a super lossy compressed copy but that’s still valuable.
- 9dev 3y agoThat’s the most dangerous possible model - produces state secrets that are mostly correct, with some juicy hallucinations about nukes, terrorists and shadow ops. What could possibly go wrong?
- lacrimacida 3y agoFake leaks
- aorloff 3y agoOr maybe just something that looks like a delayed (since this is declassification) look at what used to be important to US foreign policy interests.
- deleted 3y ago[deleted]
- jpk 3y agoIt kind of underscores the need to run training and inference on hardware you can trust, doesn't it?
- preisschild 3y agoDoes it really matter if the ex-president brags about top secret info to his golf club customers?
- TeMPOraL 3y agoObviously yes, in the same way you spilling some company secret to me over drinks is an oopsie, while me having continuous access to your corporate mailbox is a problem (for you).
- deleted 3y ago[deleted]
- googoogaga1 3y ago2023 been full of oopsies & MITM for you too :( based on anecdotal evidence I think after companies reach a certain threshold it really won't matter they compartmentalize to the max and even if something of value comes out of mailbox they probably have enough resources to track down people if needed (if they can do that I'm sure the government can figure something out)
- kortilla 3y agoWhy have medicine when you can still get paper cuts?
- Drakim 3y agoMore like, why take antibiotics if you are currently underwater and drowning.
- mistrial9 3y agoas a US taxpayer, this is an offensive comment that implies entire "secret service" compute facilities, source code and data sources. The USA was founded by people who had experienced the end games of this -- government data is PUBLIC and secret service can come back to congress and the courts each year to be reviewed. The cell phone debacle has emboldened "secret service" pension collectors with dreams of grandeur ? Peter Thiel is the new savoir ? I vote. ps- there is no secret about the population explosion of data centers near Salt Lake City, Utah or Virginia. Things are not OK.
- Obscurity4340 3y agoIs there like a JSON or like code representation one can look at that represent or portray what these weights "look" like?
- ailef 3y agoThey are just a bunch of floats, and they are usually stored in a binary non-human-readable format.
- Obscurity4340 3y agoYa but can it be easily rendered in such a fashion that humans could read it? Must be, like minified javascript or sumfing
- Drakim 3y agoSure, but they would still just be a whole long list of floats. There wouldn't be any context based on property names to gleam out of the JSON, just a long long long long list of floats.
- Obscurity4340 3y agoWhat is it keying that to tho? Like at some point, there's has to be some flowchart that integrates the floats/ratios whatever to certain human readable concepts/keywords?
- Drakim 3y agoThis is an extremely simplified explanation but basically it just multiplies the floats with each other (plus some conditional logic) and that's that, the numbers that it ends up with are translated to letters* It's surprisingly similar to how human brain neurons work, and it's not something you can translate into "hard logic". That was the old school style of doing AI and it's fallen pretty flat on it's face. Old chess AIs like Stockfish are the sort where you could examine it's logic and see how it's reasoning. These new AIs are more like a mush of mathematical operations that have been fine-tuned until the mush starts to produce viable results. That mush isn't something a human can understand by looking at it, it's just random looking numbers for miles and miles. *tokens not letters, so something like the word apple might be app + le
- MikeTheGreat 3y agoGenuine question: why would they be dangerous? By 'model weights' I assume you mean the weights on a neural net, yes? (I skimmed the article and didn't see, er, any technical details, but I could have missed something :) ). I thought that at this point pretty much nobody knows what neural net weights mean - it's kind of a 'math soup' that results from back-propagating adjustments to your huge linear equation based on training data. That said - you're not the only person here saying this / agreeing with this. What am I missing? :)
- CGamesPlay 3y agoYou use more math soup that gets the original examples out. Not perfectly, but possibly good enough. Reconstructing Training Data from Trained Neural Networks https://arxiv.org/abs/2206.07758 https://arxiv.org/abs/2206.07758
- MikeTheGreat 3y agoHuh. I never would have expected that. Every day I learn something new, I guess :) Thanks for the link!
- gonzo41 3y agoNot really. There's no way they don't get read by a human first. This is just speeding up the judgement call.