4 ms·
Thought they were gonna go CPU based inference with llama.cpp/ggml, this project should definitely be made into a programming book
by iAkashPaul 3y ago
Thought they were gonna go CPU based inference with llama.cpp/ggml, this project should definitely be made into a programming book
- NicoJuicy 3y agoI think they quickly reached the limits of CPU inference and that's why they decided to scale up with GPU's. Because of how they operate, they can't just release in a single DC like other providers. It's a partial "all or nothing" scenario.
- deleted 3y ago[deleted]