6 ms·
That's not true, there are some very pruned (and relatively dumb) LLMs which go under a gig.
by loufe 3y ago
That's not true, there are some very pruned (and relatively dumb) LLMs which go under a gig.
- andy_ppp 3y agoWhat’s the point of a state of the art AI chip that can’t run large models? It seems problematic to say the least!
- Jensson 3y agoIt can run small models very fast.
- superjan 3y agoIt is a new design, not a direct competitor for nVidia’s flagship. And there are lots more applications for AI than LLMs.
- nomel 3y agoRealtime, low power, isn't really possible today. Think anything motion or reaction related.
- jacobgorm 3y agoSome of us still work on computer vision. 224MiB is fairly massive for convolutional neural network.
- rubatuga 3y agoSmall LLMs or large small language models???