> Does anyone how an NPU would differ from a graphics card? Compare NVIDIA to Groq TSP (Tensor Streaming Processor) This is how an AI chip should be designed,

> Does anyone how an NPU would differ from a graphics card? Compare NVIDIA to Groq TSP (Tensor Streaming Processor) This is how an AI chip should be designed, you write the neural net compiler first, then design the chip. These guys have combined a number of smart ideas in their product: synchronous operation makes memory and network operations simpler to plan and optimise; all software defined memory access, no caches; a simple set of primitive operations that can express every model architecture - so they can simply make it pytorch compatible, no need to write kernels, no need to have 100 versions of 2D conv, for all sizes and shapes. https://groq.com/groq-tensor-streaming-processor-architecture-is-radically-different/ https://groq.com/groq-tensor-streaming-processor-architectur... Groq recently demonstrated 240T/s for LLaMA-2 70B. Probably world record, the next best I know is 40T/s at labs.perplexity.ai https://youtu.be/hk4QGpQAvSY?t=57 https://youtu.be/hk4QGpQAvSY?t=57