3 ms·
Care to explain? Are they using 10x energy for 10x speed improvements?
by mrtksn 3y ago
Care to explain? Are they using 10x energy for 10x speed improvements?
- wmf 3y agoThey're using hundreds of chips. Based on the data sheet I would estimate this demo uses 173 KW. It may be 100x energy to get 10x speedup.
- mrtksn 3y ago100s of chips for who knows how many clients. The mobile phone will have to do calculations just for 1 client.
- tome 3y agoYes, we pipeline requests so multiple users are being handled by the same hardware at one time.
- mrtksn 3y agoThanks for the clarification. So, would you say that Groq has a potential to have let's say OpenAI speeds on handheld devices at reasonable energy consumption? Or is that not really what this tech's strength is maybe?
- tome 3y agoThe industry as a whole is a very long way away from that. The power requirements are too high for mobile.
- pptr 3y agoI think the limitation is chip size/cost. SRAM is a lot less dense than RAM. According to Google this is typically used for registers and caches, which are only megabytes large.