3 ms·
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyw
by treprinum 2y ago
MKL runs on Ryzens just fine and newest Ryzens all have AVX512 which all new desktop/laptop Intels lack. "AI" on CPU is a gimmick, everybody does it on GPU anyway (outside 128GB RAM M3 Max that has some LLM use, but it's still due to shared memory with GPU).
- Der_Einzige 2y agoCertain workloads aren't total "gimmicks" (i.e. small embeddings models), and often those CPU optimizations are on things that absolutely do take advantage of the CPU well (i.e. graph analysis) or on traditional ML algorithms like random forests (intel optimizations for sklearn) which are still important and do still run well on CPUs. Not to mention that most people have more RAM than they do VRAM. Running MKL on your datacenter grade AMD processor technically violates some license and it relies tricks like this - https://danieldk.eu/Posts/2020-08-31-MKL-Zen https://danieldk.eu/Posts/2020-08-31-MKL-Zen Technically, if your found to be allowing this as a cloud provider by Intel, they have grounds to sue you, so yes, you can use MKL on a Ryzen at home, but you put yourself at risk for lawsuits by doing this in a data center at scale
- 0xDEADFED5 2y agoI'm doing fast AVX512 embeddings on Ryzen, and fast ONNX AVX512 reranking on Ryzen. Though I do the actual heavy lifting on GPU, doing all the RAG stuff in CPU is helpful. AI on CPU is still mostly a gimmick, but as models get smaller and more capable it's becoming less of a gimmick.
- treprinum 2y agoYeah, but the "AI" on CPU means some basic NPU doing (sparse) matrix multiplication, not AVX512.
- 0xDEADFED5 2y agoI'm handling dense, sparse, and colbert vectors generated by BGE-M3 model on CPU, pretty sure that counts as "AI"