2 ms·
Whether inference hardware will be a commodity remains to be seen. We had twenty years of most servers being x86-64. That's commodity compute. Inference hardwar
by Animats 2mo ago
Whether inference hardware will be a commodity remains to be seen. We had twenty years of most servers being x86-64. That's commodity compute. Inference hardware isn't a mature architecture yet. There will be progress and obsolescence.
- asdf88990 2mo agoIt will be in less than 5 years. You can already buy second hand RTX3090 TIs and with most of modern Macs being able to run some mind of AU locally already, in 5 years, things will be very commodified once you get the Chinese start rolling out Cuda-compatible GPUs.