3 ms·
I wonder how google runs AI on a phone doing imaging stuff and I can’t even get it to work on a 64GB RAM machine with an Nvidia 3070 card.
by spandextwins 3y ago
I wonder how google runs AI on a phone doing imaging stuff and I can’t even get it to work on a 64GB RAM machine with an Nvidia 3070 card.
- stavros 3y agoWhy doesn't it work? Try Ollama, or llama.cpp, they should work fine out of the box. They do on my 3070 just fine. Also, try a model that fits your VRAM.
- jebarker 3y agoMost the AI doing image processing is architecturally very different and smaller than LLMs.
- woadwarrior01 3y agoRunning (quantised) LLMs (encoder-decoder and decoder only transformers) on phones isn't hard. Running CNNs and encoder-only transformers, the kind used in vision on phones is even easier.