5 ms·
It's pretty simple. GPT models are essentially information weapons. People are going to get their hands on them, so might as well give them a model where you ca
by daenz 4y ago
It's pretty simple. GPT models are essentially information weapons. People are going to get their hands on them, so might as well give them a model where you can identify content generated with them, so you can know who is using them for nefarious purposes. Like how many printers encode hidden patterns on paper that identify the model of the printer and other information[0]
0. https://www.bbc.com/future/article/20170607-why-printers-add-secret-tracking-dots https://www.bbc.com/future/article/20170607-why-printers-add...
- p1esk 4y agoHow can you identify content generated with them?
- daenz 4y agoBy training a GAN. A trained GAN will be able to accurately guess whether a block of text was produced by this GPT model, some other GPT model, or is authentic.
- dotnet00 4y agoIf differentiating between real samples and generated ones were as straightforward as "training a GAN", detecting deep fakes would not be as big of a research topic as it is.
- daenz 4y agoThe point is that it's possible and we're improving on it every day.
- onethought 4y agoJust so I understand you properly: Original Inputs (A) -> NN (Q) -> Output (X) You are saying you could train something that would take X and identify that it is the product of NN (Q). Even though you don't know A? So, to simplify and highlight the absurdity: If I made a NN that would complete sentences by putting a full stop on the end of open sentences. You could train something that could detect that separately to a human placed full stop? (This seems actually impossible, there is an information loss that occurs that can't be recovered)
- daenz 4y agoCan you identify GPT text versus authentic text? If so, then there are features in that text that give it away. It stands to reason that there exist other features in the text, based on the training data the model was fed, and other characteristics of the model, that a discriminator model could use to detect, with some confidence, which model produced the text. A discriminator model which can detect a specific generative model essentially captures its "fingerprint". An example of some of these features might be the use of specific word pairs around other word pairs. Or a peculiar verb conjugation in the presence of a specific preposition.
- adamsmith143 4y agoKnow any papers where someone has done this with large language models successfully?
- PeterisP 4y agoI'm not saying that Meta did it, but recent research shows that it is possible and hard to detect - https://arxiv.org/abs/2204.06974 https://arxiv.org/abs/2204.06974 - so if they really wanted to, they could.
- orbital-decay 4y agoThat paper is not about fingerprinting the arbitrary output of a specific model, which would allow Meta to track its usage in the results, e.g. tell a genuine text from a fake generated by their model. The paper implies giving the model some specific secret input only known to you. I think the thread we're in is also based on the similar misunderstanding.
- voz_ 4y agoThis is nonsense.
- daenz 4y agoWould an AI @ FB employee admit it if it was true?
- voz_ 4y ago> I will never discuss FB technical details, internals, or anything else on this site, so please do not ask. My claim of nonsense has nothing to do with FB. You cannot fingerprint models like this, that's just not how it works. Also, if we are reading profiles, you call yourself a 10x engineer on your blog, that's hilarious. Maybe 10x the nonsense?
- daenz 4y ago>You cannot fingerprint models like this A GAN can absolutely be trained to discriminate between text generated from this model or another model. >that's hilarious What's hilarious about it?
- voz_ 4y ago> A GAN can absolutely be trained to discriminate between text generated from this model or another model. Nope. I dare you to do it. Or at least intelligently articulate the model architectures for doing so. > What's hilarious about it? It's a bullshit term, firstoff, and calling yourself that is the height of ego. Might as well throw in rockstar, ninja, etc too.
- daenz 4y agoSo in the entire field of machine learning, we can't train a model that can identify another model from its output? Just can't be done? And there's absolutely no value in having tools that can identify deep fakes, or content produced by specific open models? >It's a bullshit term, firstoff, and calling yourself that is the height of ego I am a 10x engineer though, so I'm sorry if that rubs you the wrong way. Also, you're reading my personal website, so of course I'm going to speak highly of myself :)