3 ms·
Are they though? They are lossy compressing trillions of tokens into a few dozen GB. The decompression action is fuzzy and inefficient though.
by samus 3y ago
Are they though? They are lossy compressing trillions of tokens into a few dozen GB. The decompression action is fuzzy and inefficient though.
- raincole 3y agoAnd it requires massive computational power to decompress, which I don't expect to be available in a catastrophic situation where humans have lost a large chunk of important knowledge.
- samus 3y agoI don't necessarily agree. It requires massive computing power, but running models smaller than 70G parameters is possible on consumer hardware, albeit slowly.
- threecheese 3y agoParent may be thinking more along the lines of a “hope we can print all the knowledge“ type catastrophe. Though if there is zero compute it’ll be tough reading all those disks!