13 ms·
If you want to build a model on English text, you have terabytes of webpages to use. Exposing sensitive information (like, the phrase "Adam, I'm sorry your cat
by lsb 16y ago
If you want to build a model on English text, you have terabytes of webpages to use.
Exposing sensitive information (like, the phrase "Adam, I'm sorry your cat has feline AIDS" compresses via LZ to position 1230352024, so your account has 1.950001GB free instead of 1.949999GB free) is the wrong way to squeeze a few more bits out; make sure you don't break through the ACL of each document.
Another feature to be mindful of is how much power you'll need for the text model; larger models compress better, but need more time and space.