3 ms·
That makes some sense. NAND Flash is massively parallel by its nature. That is rarely exposed outside the die though. You'll have that 8 bit double data rate bu
by ACCount37 3mo ago
That makes some sense. NAND Flash is massively parallel by its nature. That is rarely exposed outside the die though. You'll have that 8 bit double data rate bus and you'll learn to like it.
Now that model inference at scale is a thing though? Model weights, cached prefixes? There's a considerable demand for "slow writes, fast high bandwidth reads" memory. And every bit of storage you didn't have to use RAM for you can use for fatter KV cache instead.