3 ms·
It all comes down to the VRAM. The average person will never have the money to buy a single H100 96gb let alone a DGX server or it's future equivalents You get
by Der_Einzige 4y ago
It all comes down to the VRAM. The average person will never have the money to buy a single H100 96gb let alone a DGX server or it's future equivalents
You get fundamentally more powers when you add more VRAM in ways that are just hard to explain to folks outside of this ecosystem. Everything around the VRAM are basically small details in comparison
- dalys 4y agoPlus, I assume people want to have their assistant on their phone, not their desktop computer. So until everything can run locally on your phone, I think people will prefer the cloud versions.
- dragonwriter 4y agoI assume some people want their assistant to work on data they don’t want to share with megacorps and governments, and some of those people can figure out how to make their phone talk securely to a home server over the internet.
- pmoriarty 4y ago"Everything around the VRAM are basically small details in comparison" ...with current algorithms and our lack of understanding and insight in to how/why they work on a deep level or what intelligence and consciousness is. With time hopefully all of these will improve and perhaps future AI's of good quality will be affordable to mere mortals.
- SanderNL 4y agoI know it’s not average person money, but the average person is not looking for hardware to run inference on LLMs. A100 seem in the €15k ballpark and H100 double that. Lot of money but I am actually surprised. A dedicated regular guy could buy this. I mean people buy cars and don’t really need them either. Again not saying it is a bargain, but it’s not billionaires only territory and that is good news (it’s early days!).