7 ms·
What, even if it means you can run models without relying on the currently backlogged DRAM production?
by trebligdivad 2mo ago
What, even if it means you can run models without relying on the currently backlogged DRAM production?
- adgjlsfhk1 2mo agoThe size of model we're talking about running doesn't need much if any dram.
- trollbridge 2mo agoThe chatjimmy demo is using a model that needs 6-18GB of VRAM. That's not exactly trivial. I could see it being feasible to get a Qwen-3.6-27b type of model done on something like this. Qwen-3.6-27b at 18tok/s would be a game changer.
- adgjlsfhk1 2mo agoright, but that's a reticle size chip. to put something in a phone it has to be ~10-30x smaller