2 ms·
> Will demand reach 1TB of RAM for a consumer level computer/mobile device? No. Speculative Sampling allows to run a smaller model locally and only check up wi
by visarga 4y ago
> Will demand reach 1TB of RAM for a consumer level computer/mobile device? No.
Speculative Sampling allows to run a smaller model locally and only check up with a larger model in the cloud from time to time, once every few tokens.