4 ms·
The implications of this sentiment are disturbing. It is considered an "attack" to probe at something to understand how it works in detail. In other words, ho
by userbinator 3y ago
The implications of this sentiment are disturbing.
It is considered an "attack" to probe at something to understand how it works in detail.
In other words, how basically all natural science is done.
What the fuck has this world turned into?
- charcircuit 3y agoDo you consider blind SQL injection an attack? If a user is not meant to have access to information, but they can somehow access it, that is a vulnerability.
- idle_zealot 3y agoSure, except the information here is not other users' account info or something, it's analogous to the source of the DBMS. Which is not typically considered sensitive.
- bugglebeetle 3y agoTell that to Oracle…
- charcircuit 3y agoNo, it's like the schema of the database which is normally a secret.
- deleted 3y ago[deleted]
- ronsor 3y agoOpenAI has blatantly said that the "open" in their name was a deceptive marketing ploy. At this point in time, they aren't interested in sharing much if any real research, and trying to discover this information is now an "attack" against them.
- astrange 3y agoOpenAI didn't write this paper.
- dr_kiszonka 3y agoOne of the co-authors is from OpenAI.
- yreg 3y ago> "open" in their name was a deceptive marketing ploy It is now, but it hasn't been since the beginning. Hence the elon lawsuit.
- hnfong 3y agoWe used to call this "reverse engineering". I'm not familiar with what the law says on reverse engineering, but calling it "stealing" seems a bit too much.
- imjonse 3y ago"Reverse engineering part of a production language model" has a more boring ring to it though.
- ixaxaar 3y agoThe United States.
- imjonse 3y agoAttack here is a term specific to cryptography and infosec, it does not imply doing anything illegal or violent. "Oracle attack" is the more specific term where you probe a system to give up some of its internals it was not meant to expose.
- TeMPOraL 3y agoWording matters. To use LLM terminology, "attack" here may mean something neutral, but it's helluva closer to "fear" and "malice" and "evil" in the latent space than terms like "probing", "studying", "examining", or "reverse engineering". FWIW, cryptography and infosec as fields get a good mileage out of exploiting fear.
- imjonse 3y agoThat is only true in everyday latent space/vocabulary. In the ones underlying arxiv papers, cryptography and LLM, attack has the latter meaning.
- TeMPOraL 3y agoPerhaps, but everyday vocabulary is what public policy and law discussions happen in, and I think that's what 'userbinator is worried about (and so am I). See also: "piracy is stealing" or the everyday vocabulary meaning of the word "hacker".
- andy99 3y agoIt's getting information the org that created and hosts the model doesn't want you to have. Just because you think that information should be shared doesn't make it any less of an attack.