4 ms·

Why? Dolly had nothing to do with Llama or its weights. Besides: How would anyone ever know which model generated the output you are serving? AFAIK there is no

by leobg 4y ago

Why? Dolly had nothing to do with Llama or its weights. Besides: How would anyone ever know which model generated the output you are serving? AFAIK there is no fingerprint in any model’s output. And even if there was, it would probably be destroyed by fine tuning “over it”.

stametseater 4y ago
> AFAIK there is no fingerprint in any model’s output. It seems like there easily could be. What if some of the data they trained it on didn't exist anywhere else except in the training set, and was put there specifically for this purpose? For instance they could have taught it a few poems that don't exist anywhere else. If you can coax the LLM of unknown origin into reciting those poems back to you, you know where it came from.
- kurthr 4y ago
  Even easier have a small set of 8-10 character gibberish tokens it's trained on in a particular contexts (eg a non-existent poem). Then feed it one or several poems and see if a gibberish token pops out.
  - eigenvalue 4y ago
    I think they call these canary GUIDs. If you manage to generate one from an LLM then you can conclude with certainty that the model saw that document during training.
neilv 4y ago
> Besides: How would anyone ever know which model generated the output you are serving? There's precedent for "whatever you can get away with" in tech companies, but establishing a culture of that at the start of this new big change could end up undesirable for most people. For example, it could relieve demand for more legal and sustainable ways, until it's too late. (Look at the history of digital entertainment media piracy and DRM and legislation, for example. Or look at the history of software piracy, where some big companies seem to actually want their product to be pirated, partly because it builds a bigger moat against competitors, and they can legally strongarm some of those pirates later.)

by leobg 4y ago

stametseater 4y ago

> AFAIK there is no fingerprint in any model’s output. It seems like there easily could be. What if some of the data they trained it on didn't exist anywhere else except in the training set, and was put there specifically for this purpose? For instance they could have taught it a few poems that don't exist anywhere else. If you can coax the LLM of unknown origin into reciting those poems back to you, you know where it came from.

kurthr 4y ago
Even easier have a small set of 8-10 character gibberish tokens it's trained on in a particular contexts (eg a non-existent poem). Then feed it one or several poems and see if a gibberish token pops out.
- eigenvalue 4y ago
  I think they call these canary GUIDs. If you manage to generate one from an LLM then you can conclude with certainty that the model saw that document during training.

neilv 4y ago

> Besides: How would anyone ever know which model generated the output you are serving? There's precedent for "whatever you can get away with" in tech companies, but establishing a culture of that at the start of this new big change could end up undesirable for most people. For example, it could relieve demand for more legal and sustainable ways, until it's too late. (Look at the history of digital entertainment media piracy and DRM and legislation, for example. Or look at the history of software piracy, where some big companies seem to actually want their product to be pirated, partly because it builds a bigger moat against competitors, and they can legally strongarm some of those pirates later.)