5 ms·
Agreed, but this is a blocker for anyone seriously considering using the product. CPU-only inference simply isn’t good enough for anything besides toy workloads
by babelfish 3y ago
Agreed, but this is a blocker for anyone seriously considering using the product. CPU-only inference simply isn’t good enough for anything besides toy workloads. If they’re waiting for people to use Constellation before investing in GPU support, nobody will use constellation because it doesn’t have it, so they’ll never end up investing in it, and so on…
- dknecht 3y agoI think people will be surprised how far CPU optimization will go for specialized inference. An example of the progress being made - http://ggml.ai/ http://ggml.ai/
- NicoJuicy 3y agoThey use it themselves to route traffic, which probably is a not gpu intensive. I've ran face detection on cpu and it takes 1 / secondv and it's probably a pretty intensive "action". Plenty of use-cases that don't require a gpu And plenty that require one too though.