3 ms·

> The compression is achieved by using the probability of the next word computed by the GPT-2 language model released by OpenAI. Does this mean that as the mod

> The compression is achieved by using the probability of the next word computed by the GPT-2 language model released by OpenAI. Does this mean that as the model evolves (I assume it's not static) there could be a chance of decompressing a blob of text and having it say something different to the original?

jerf 6y ago
It is static.
sdenton4 6y ago
Yes. Neural compression requires a combination of a model and a compressed message. In the future, my guess is that (some) compression protocols may end up pretty static, and mainly consist of a simple message protocol and a way to check that you've got the correct version of the model for the messages you want to decompress.
jkhdigital 6y ago
The trained model parameters are effectively the coding dictionary. Changing the parameters is like using Huffman compression but sending a different Huffman tree along with the compressed text—of course it will decompress to something different.
MiroF 6y ago
> I assume it's not static Why? It is static.
- BuildTheRobots 6y ago
  Because I naively assumed the strength of an AI model would mean it would evolve to best match/understand your specific input (ala classical custom dictionaries or even just personalised models ala swiftkey) but I've obviously entirely misunderstood what's going on :\
  - whimsicalism 6y ago
    That would require distributing the trained AI model weights with your compressed text, entirely defeating the space gains for all but extremely large passages of text.
gorkish 6y ago
Sure; this would be the equivalent of a "lossy" compression algorithm which encodes the result in terms of a question using semantics related to the respondent's knowledge to produce a specific response. Anyone who has been through elementary school should understand this is the basic objective of a "short answer" question. Also "Jeopardy" is a fair example of this.

jerf 6y ago

It is static.

sdenton4 6y ago

Yes. Neural compression requires a combination of a model and a compressed message. In the future, my guess is that (some) compression protocols may end up pretty static, and mainly consist of a simple message protocol and a way to check that you've got the correct version of the model for the messages you want to decompress.

jkhdigital 6y ago

The trained model parameters are effectively the coding dictionary. Changing the parameters is like using Huffman compression but sending a different Huffman tree along with the compressed text—of course it will decompress to something different.

MiroF 6y ago

> I assume it's not static Why? It is static.

BuildTheRobots 6y ago
Because I naively assumed the strength of an AI model would mean it would evolve to best match/understand your specific input (ala classical custom dictionaries or even just personalised models ala swiftkey) but I've obviously entirely misunderstood what's going on :\
- whimsicalism 6y ago
  That would require distributing the trained AI model weights with your compressed text, entirely defeating the space gains for all but extremely large passages of text.

gorkish 6y ago

Sure; this would be the equivalent of a "lossy" compression algorithm which encodes the result in terms of a question using semantics related to the respondent's knowledge to produce a specific response. Anyone who has been through elementary school should understand this is the basic objective of a "short answer" question. Also "Jeopardy" is a fair example of this.