3 ms·
A GPU-less machine? I've rented a server but it has no GPU. Does MLC work well through only CPU inference? I'd like to get it set-up with langchain if it does
by cpullm 3y ago
A GPU-less machine?
I've rented a server but it has no GPU. Does MLC work well through only CPU inference?
I'd like to get it set-up with langchain if it does work well
- junrushao1994 3y agoTVM Unity, the compiler used by MLC-LLM, does support CPU and SIMD instructions on each CPU backend via LLVM, but we haven't tried it out yet. I believe llama.cpp is the best option out of box at the moment.