2 ms·
It's intended for SQL generation and similar with cheap fine tuning and inference, not answering general knowledge questions. Their blog post is pretty clear ab
by mike_hearn 3y ago
It's intended for SQL generation and similar with cheap fine tuning and inference, not answering general knowledge questions. Their blog post is pretty clear about that. If you just want a chatbot this isn't the model for you. If you want to let non-SQL trained people ask questions of your data, it might be really useful.
- mritchie712 3y agoIt's worse at SQL generation than llama3 according to their own post. https://www.snowflake.com/blog/arctic-open-efficient-foundation-language-models-snowflake/#:~:text=on%20enterprise%20intelligence.-,Table%203,-.%20Full%20Metrics%20Table https://www.snowflake.com/blog/arctic-open-efficient-foundat...
- CharlesW 3y agoTo be fair, that's comparing their 17B model with the 70B Llama 3 model.
- ru552 3y agoTo stay fair, their "17B" model sits at 964GB on your disk and the 70B Llama 3 model sits at 141GB. unquantized GB numbers for both
- CharlesW 3y agoSorry, it sounds like you know a lot more than I do about this, and I'd appreciate it if you'd connect the dots. Is your comment a dig at either Snowflake or Llama? Where are you finding the unquantized size of Llama 3 70B? Isn't it extremely rare to do inference with large unquantized models?
- fsiefken 3y agoto stay fairer, the required extra disk space for snowflake-arctic is cheaper then the required extra ram memory for llama3
- Manabu-eo 3y agoFor decent performance, you need to keep all the parameters on memory for both. Well, with a raid-0 of two PCIe 5 SSDs (or 4 PCIe 4) you might get 1 t/s loading experts from disk on snowflake-artic... but that is slooow.