3 ms·

Few gigabytes? The smallest ones quantized to where you have to squint a little. 70B weights at fp32 is a lot of Wikipedia. GPT-4 is rumored to be 1.5 terrawe

Few gigabytes? The smallest ones quantized to where you have to squint a little. 70B weights at fp32 is a lot of Wikipedia. GPT-4 is rumored to be 1.5 terraweights. There’s compression but it’s not magical.

chpatrick 3y ago
Even 6TB is a couple garden variety hard drives, that's not much compared to the internet.
- usrnm 3y ago
  What if "the internet" is just the same stuff under slightly dufferent dressings, and LLMs are good at seeing through it?
  - chpatrick 3y ago
    That's exactly my point.
  - notnaut 3y ago
    Well, yeah. In the same sense that all songs are just the same stuff dressed differently because there are finite musical notes. Doesn’t really take away from the majesty of music.