4 ms·
Training LLM models and generating LLM responses is probably more expensive than... basically anything else we have now.
by vdaea 3y ago
Training LLM models and generating LLM responses is probably more expensive than... basically anything else we have now.
- ldjkfkdsjnv 3y agoIts expensive right now. Compute cost always drops like a rock
- refulgentis 3y agoIt's quietly broken in favor of local LLMs over the past month. I say this as someone who was a huge skeptic until maybe 2 weeks ago. It's not GPT-4 but it doesn't need to be. StableLM 3B can handle RAG inputs. This is huge. 7B models can't run on consumer mobile hardware: https://x.com/jpohhhh/status/1747451790969184682?s=20 https://x.com/jpohhhh/status/1747451790969184682?s=20 llama.cpp got examples for iOS / Android in December. It can run on all platforms via one library: https://x.com/jpohhhh/status/1748852554920849579?s=20 https://x.com/jpohhhh/status/1748852554920849579?s=20 retrieval/vector DB can run on all platforms via one library: https://github.com/Telosnex/fonnx https://github.com/Telosnex/fonnx