4 ms·

I think you might be confusing the GPT software (a generative pre trained transformer) with the finished product, an LLM (large language model.) A GPT has no t

I think you might be confusing the GPT software (a generative pre trained transformer) with the finished product, an LLM (large language model.) A GPT has no training until you give it materials. I do believe Google released the code for theirs ages ago. Even without source, you can run a GPT against your own data locally, or on a cloud service setup for that purpose. This is how Bloomberg, for example, created a financial LLM. They used a GPT to train on their own financial data.

moneywoes 4y ago
Any examples of doing that process cost effectively?
- tough 4y ago
  Not what you're asking but Vicuna did cost merely 300$ to fine-tune on top of LLaMA https://www.marktechpost.com/2023/04/02/meet-vicuna-an-open-source-chatbot-that-achieves-90-chatgpt-quality-and-is-based-on-llama-13b/ https://www.marktechpost.com/2023/04/02/meet-vicuna-an-open-... AFAIK full model training should be a couple order magnitudes higher probably?
- dtagames 4y ago
  For many projects, you'll need "natural language" training on regular text documents in order to be able to process even your prompts. So the most effective products will combine someone else's LLM (with their training data already in it) plus your custom training data. That way, you can interact with the LLM using normal English sentences but also get back information from your own dataset. Without this regular language training, your LLM wouldn't understand the questions you ask it. So there are two cost factors... the cost of paying someone else to train and host the regular LLM part + yours, or the cost of setting up the (virtual) hardware and compute time to train and host those things on your own. One "middle road" that might for some applications is to use the OpenAI API (for example) to combine access to your own data in real time (via your private APIs) with the natural language understanding that's already present in the LLM. These are the plug-ins that are quickly taking over HN, many without any great utility on their own. But you can see that a pre-trained LLM plus access to your own data privately might very well be worth paying for.

moneywoes 4y ago

Any examples of doing that process cost effectively?

tough 4y ago
Not what you're asking but Vicuna did cost merely 300$ to fine-tune on top of LLaMA https://www.marktechpost.com/2023/04/02/meet-vicuna-an-open-source-chatbot-that-achieves-90-chatgpt-quality-and-is-based-on-llama-13b/ https://www.marktechpost.com/2023/04/02/meet-vicuna-an-open-... AFAIK full model training should be a couple order magnitudes higher probably?
dtagames 4y ago
For many projects, you'll need "natural language" training on regular text documents in order to be able to process even your prompts. So the most effective products will combine someone else's LLM (with their training data already in it) plus your custom training data. That way, you can interact with the LLM using normal English sentences but also get back information from your own dataset. Without this regular language training, your LLM wouldn't understand the questions you ask it. So there are two cost factors... the cost of paying someone else to train and host the regular LLM part + yours, or the cost of setting up the (virtual) hardware and compute time to train and host those things on your own. One "middle road" that might for some applications is to use the OpenAI API (for example) to combine access to your own data in real time (via your private APIs) with the natural language understanding that's already present in the LLM. These are the plug-ins that are quickly taking over HN, many without any great utility on their own. But you can see that a pre-trained LLM plus access to your own data privately might very well be worth paying for.