4 ms·
You have a weird definition of "good model" Llama 7B is NOT a good model.
by throwawayadvsec 3y ago
You have a weird definition of "good model"
Llama 7B is NOT a good model.
- chaxor 3y agoYou can run much larger models than llama-7B. Galpaca-30b or Galactica-120b for example.
- int_19h 3y ago30B is still not good enough. What kind of desktop are you running a 120B model on with reasonable performance?
- chaxor 3y agoI would disagree that 30B is not good enough. It heavily depends on which model, and what you're trying to use it for. 30B is plenty if you have a local DB of all of your files and wiki/stackechange/other important databases places in a embedding vectordb. This is typically what is done when people make these models for their home, and it works quite well while saving a ton of money. While llama-7B systems on their own may not be able to construct a novel ML algorithm to discover a new analytical expression via symbolic regression for many-body physics, you can still get a great linguistic interface with them to a world of data. You're not thinking like a real software engineer here - there are a lot of great ways to use this semantic compression tool.
- int_19h 3y agoIf I just wanted a fuzzy search engine for local data, I'd use a vector DB - there's no need for an LLM on top of that.