2 ms·
There are huge differences. I used OpenAI’s APIs in my last book on LangChain and LlamaIndex, and GPT-3.5 and GPT-4 are good enough right now to support buildin
by mark_l_watson 3y ago
There are huge differences. I used OpenAI’s APIs in my last book on LangChain and LlamaIndex, and GPT-3.5 and GPT-4 are good enough right now to support building applications for years (although I look forward to improvements).
I am writing a new book Safe For Humans AI in which I am constraining myself to using open models that can be run on a high end PC or a leased GPU server. Yesterday I was exploring what I could do with T5-flan-XXL, and it is useful, but not as effective tool as the closed OpenAI models. I will do the same with Orca this week.
- espadrine 3y agoI think a lot of people see a larger gap between open-source models and GPT-4 than is present because they test out the 7B models which fit on their machine. Models like T5-FLAN-XXL are very far below the quality one can expect from the best in open-source, and barely usable for CoT or tool use. Especially for LangChain, I recommend using ≥33B models like Guanaco or WizardLM. Guanaco-65B honestly feels on-par with ChatGPT-3.5. (To be clear, there is a large gap with GPT-4 though.) It is a costly test, although GPTQ (for instance on exllama) help make it affordable. I haven’t tried Orca since they haven’t released the weights yet, but it doesn’t seem like they have a 33B version.
- mark_l_watson 3y agoThanks for your advice! I am trying to write the book examples using a A6000 (48G video memory), but I may have to go higher end. The hourly lease charge on a A6000 is very inexpensive and I wanted the examples to be "almost free" to run. I will see what is required for Guanaco-65B.