4 ms·
> new generations of AI focused hardware Some benchmarks are not encouraging. See e.g. https://www.hardware-corner.net/mac-studio-m3-ultra-deepseek-llamacpp/ h
by mdp2021 2y ago
> new generations of AI focused hardware
Some benchmarks are not encouraging. See e.g. https://www.hardware-corner.net/mac-studio-m3-ultra-deepseek-llamacpp/ https://www.hardware-corner.net/mac-studio-m3-ultra-deepseek...
That «AI focused hardware» will either have extremely fast memory, and cost prohibitively, or have reasonable costs, and limits that are to be assessed.
- andrewstuart 2y agoErrrr that’s a 671B model.
- mdp2021 2y agoYes, but what will you need as you will prepare to be set for your personal needs? We are far from having reached optimal technology at trivial cost. State-of-the-art commercial VRAM is over 10x faster than the standard one - and costs well over 10x. Reasonably available speeds may or may not be acceptable.