29 ms·
Ah ok, I think we made different assumptions about whether the model was specific to the particular dataset so each one would need a new model — a dictionary is
by binary132 2y ago
Ah ok, I think we made different assumptions about whether the model was specific to the particular dataset so each one would need a new model — a dictionary is specific to the particular dataset being compressed, right? I was thinking the LLM would be a general-purpose text compression model.
- remram 2y agoNot particularly. You could make a dictionary from "the English web", with common character sequences found on those sites you use as input.