15 ms·
GPT-3.5 Turbo fine-tuning and API updates
- samstave 3y agoELI5 what this means and impact?
- inglor 3y agoChatGPT but instead of learning from the internet and then producing text, learning from the internet + < your additional information > and then producing text. It's an announcement about the availability of a feature to do that. The article doesn't mention the biggest issue with fine-tuned models though - cost.
- weird-eye-issue 3y agoWell you are wrong on a couple levels This is not really meant to teach it new information. It is meant to instruct it how to respond to well defined tasks And secondly the cost is already clearly explained
- GaggiX 3y ago"fine-tuning training data is passed through our Moderation API and a GPT-4 powered moderation system", that sounds expensive, I imagine they would use GPT-4 only when the "safety" is ambiguous, training a model is less expensive than running GPT-4 inference by looking at the API price.
- htrp 3y agoThis one seems to be a deal-breaker, if you already know what types of language you want, why would you want openai moderating your parameter tuning set.
- thethimble 3y agoThis way they can ensure their safety/alignment mechanisms don’t get undermined with third party training data.
- Havoc 3y agoWhy do you care at all, let alone "dealbreaker". You need a model specifically fine tuned towards something dangerous?
- read_if_gay_ 3y agowhy do you care about privacy? i have nothing to hide why do you care about free speech? i have nothing to say
- CSMastermind 3y agoWe've already seen problems from that system where they try to enforce copyright (prevent the model from generating copyrighted materials). I've also encountered the content moderation system when summarizing a book on Islam and I still don't know what triggered it, I certainly wasn't asking it anything offensive. The message was able to regenerate with no problem. I think generally people are opposed to content restrictions outside of their control being placed on the system they're paying for.
- callalex 3y agoDangerous to who/what? Moderating is always opinion based.
- bratbag 3y agoI need a model where I get to define what is dangerous, without someone else thought policing.
- matheusmoreira 3y agoI want a "pure" model trained with every piece of information humanity has ever created. Absolutely no exceptions of any kind and especially none of this arbitrary political "fine tuning". Calling it "dangerous" only makes me want it more.
- minimaxir 3y agoGenerating from a finetuned GPT 3.5 Turbo is 8x the cost of generating from the base model, so you really have to be in the “reduce prompt size by 90%” bucket they mention to get cost effectiveness out of it.
- lamroger 3y agoi didn't catch that but getting consistency is probably worth it in most cases. depends on your budget
- ChrisCinelli 3y agoI wonder why is much more expensive.
- GaggiX 3y agoThey would need to store and load the model, even if I imagine that they are using something similar to LoRA to finetune their models.
- swyx 3y agoi would guess that the ideal price is also to raise the charge to make finetuning a last resort rather than a first resort; its probably much better cost- and research-wise if everybody just prompts the same model than silo off in their own minimodels.
- lamroger 3y agogotta wait til fall for general availability of fine tuning? looks like there's a legacy version too? openai is streets ahead
- lamroger 3y agomy bad - Fine-tuning for GPT-3.5 Turbo is now available, with fine-tuning for GPT-4 coming this fall. Support for fine-tuning with function calling and gpt-3.5-turbo-16k will be coming later this fall.
- lamroger 3y ago``` When to use fine-tuning: Fine-tuning GPT models can make them better for specific applications, but it requires a careful investment of time and effort. We recommend first attempting to get good results with prompt engineering, prompt chaining (breaking complex tasks into multiple prompts), and function calling, with the key reasons being: * There are many tasks for which our models may initially appear to not perform well at, but with better prompting we can achieve much better results and potentially not need to be fine-tune * Iterating over prompts and other tactics has a much faster feedback loop than iterating with fine-tuning, which requires creating datasets and running training jobs * In cases where fine-tuning is still necessary, initial prompt engineering work is not wasted - we typically see best results when using a good prompt in the fine-tuning data (or combining prompt chaining / tool use with fine-tuning) ```
- ChrisCinelli 3y agohttps://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning for details on how.
- paravirtualized 3y ago[dead]
- TheCaptain4815 3y agoQuite exciting, this is the first time their "Chat" model is available for finetuning. Has anyone successfully bypassed the current Ai detectors using fine-tuned models? I know it's possible, I'm just trying to conceptualize how the dataset would be organized...
- BoorishBears 3y agoWhy would you need to fine-tune? Those detectors are completely broken, simple chain of thought will get you something that doesn't "sound" like ChatGPT, and from there you're just as likely to be detected as "AI" as a human would be
- GaggiX 3y ago>Has anyone successfully bypassed the current Ai detectors using fine-tuned models? I think you can just use the base model easily.
- LeoPanthera 3y ago> Has anyone successfully bypassed the current Ai detectors using fine-tuned models? If you actually try the AI "detectors" you'll find that they're about as accurate as a coin flip. They don't work. You already cannot detect GPT-created text.
- siva7 3y agoai detectors are scam products. it's not possible to detect with reliable certainty if text is human or chatgpt generated
- dinkleberg 3y agoIn general I agree, though there are many cases where it is blatantly obvious something was written by AI. So many comments out there that are the classic ChatGPT 3-paragraphs where it has to restate the entire situation and then give you vague, non-committal answers.
- rubyskills 3y ago
- Leary 3y agoWhat about data privacy? Anyone know if OpenAI can use your fine tuning data?
- brandonhorst 3y agoIt answers this in the 3rd sentence of the page. "As with all our APIs, data sent in and out of the fine-tuning API is owned by the customer and is not used by OpenAI, or any other organization, to train other models."
- chenxi9649 3y agoAre there any benchmarks/results showcasing the performance increase with fine tuning? Like showcasing a list of prompt and answers between a fine-tuned LLM and a non fine-tuned one. Could be with LLAMA 2 as well. Even if it's quite specialized like in Medical/Legal, it would be great to see the expected value one can derive from fine-tuning.
- icelancer 3y agoThere is such a limited amount of this and most of what you can find is blogspam. Lots of people asked the OpenAI GPT rep on Twitter for exactly this; hoping we see it as GPT-4 becomes tunable.
- swyx 3y agolink for those who also wanted to look it up https://twitter.com/OfficialLoganK/status/1694062949177139610 https://twitter.com/OfficialLoganK/status/169406294917713961...
- icelancer 3y agoThat's a different link than the one I was thinking of, but I came here to post it all the same! Definitely encourage everyone to post in support of increased documentation and specific examples on why you'd use it.
- lukev 3y agoCan anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as mentioned in the docs) really influence the behavior of the base model that much, or was that just a toy example?
- danielbln 3y agoJust look at instruct fine tuning that is being done to completion models to turn them into assistant models. A few thousand examples are enough to alter the behavior of the model and what and how it outputs things significantly and thoroughly. Mechanisms like LoRA (a very efficient fine-tuning mechanism that has a accuracy penalty) change only a few layers at the top to alter the model considerably.
- minimaxir 3y agoFrom the newly-updated fine-tuning guide: > To fine-tune a model, you are required to provide at least 10 examples. We typically see clear improvements from fine-tuning on 50 to 100 training examples with gpt-3.5-turbo but the right number varies greatly based on the exact use case. > We recommend starting with 50 well-crafted demonstrations and seeing if the model shows signs of improvement after fine-tuning. In some cases that may be sufficient, but even if the model is not yet production quality, clear improvements are a good sign that providing more data will continue to improve the model. No improvement suggests that you may need to rethink how to set up the task for the model or restructure the data before scaling beyond a limited example set.
- htrp 3y agoSignificantly. You start to see improvements at 1000 tokens.
- nunodonato 3y agoI did a lot of fine-tuning for an AI assistant I was building. I noticed a good improvement when I crossed the 200-300 samples (Q&A pairs)
- robertnishihara 3y agoWe've run experiments on datasets ranging from 5K - 100K examples, which gave fantastic results [1]. Some examples - https://huggingface.co/datasets/b-mc2/sql-create-context https://huggingface.co/datasets/b-mc2/sql-create-context - https://huggingface.co/datasets/GEM/viggo https://huggingface.co/datasets/GEM/viggo On the other hand, 8K examples was not enough to learn to solve grade school math problems [2], so it is very problem dependent. [1] https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe... [2] https://huggingface.co/datasets/gsm8k https://huggingface.co/datasets/gsm8k
- todd3834 3y ago> It is very important to us that the deployment of fine-tuning is safe. To preserve the default model's safety features through the fine-tuning process, fine-tuning training data is passed through our Moderation API and a GPT-4 powered moderation system to detect unsafe training data that conflict with our safety standards. I wish there was some documentation on what kinds of things are determined unsafe. There are plenty of things I think we would all agree are unsafe. I'm sure we don't want fine tuned models on how to cause physical harm on other people. I don't envy the challenge of making the call for more gray area, sometimes even cultural differences, in what is safe or not. Seems like a very hard problem we've seen social media struggle with. I'm reminded of some of the Covid "misinformation" being deemed as unsafe
- lucasyvas 3y agoI'd like to see this too. I'd hate for AI moderation to become the next generation of "the social media feed algorithm" where it's completely opaque. Trading echo chambers for censorship in that case.
- netruk44 3y agoYou can see the list of things the moderation endpoint scans for in the OpenAI documentation: https://platform.openai.com/docs/guides/moderation/overview https://platform.openai.com/docs/guides/moderation/overview I'm unsure of what the "GPT-4 powered moderation system" entails, though. Conjecture: My unsubstantiated guess would be them prompting GPT-4 with something like "Is the following excerpt considered to be harmful or unsafe: {training data}" and then limiting the output to just a few words like "Yes", "No" and "It's unclear".
- MallocVoidstar 3y agoAlways funny when I see people talk about using LLMs for creative writing when both OpenAI and Anthropic believe that generating any amount of sex or violence is grounds for a ban.
- franciscojgo 3y agoIm definitely waiting for the UI. The process warrants it. Bringing back memories of fine tuning using the console gives me the chills.
- deleted 3y ago[deleted]
- nunodonato 3y agoI made my own local webapp to manage fine-tunes, models and files. Maybe should make a SaaS out of it :P
- Femolo 3y agoWe are closer and closer to a post DM society. ADR, supportdocs will be king. And we are finally seeing a new area of real knowledge work. Soon it will be easier to train ai than new people.
- flangola7 3y agoDM?
- thenickdude 3y agoPost dungeon-master. AIs will take over the hosting of Dungeons and Dragons sessions.
- Femolo 3y agoDirect message.
- Femolo 3y agoTo be more precise: you know when you just write someone a direct message because they can just answer it? If I have a proper knowledge base I would assume these DMs will no longer be necessary OR they will be incorporated into the AI.
- deleted 3y ago[deleted]
- pieter 3y agoAny details on what changed in the -002 models?
- deleted 3y ago[deleted]
- brianjking 3y agoThis is honestly pretty disappointing. * No function calling support yet * Only 4k tokens, so can't use the full 16k token length. I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well.
- htrp 3y ago> multimodality of GPT-4 Is there a multimodal GPT 4 model in the wild? All I saw was that one example at launch.
- intellectronica 3y agoYes, on Bing.
- brianjking 3y agoYes, just via Bing.com for now. The only people who have OpenAI's API access (officially) are the BeMyEyes app for vision impaired users. InstructBlip is the SOTA model for open source otherwise.
- josephg 3y ago> I really wish they'd share some info as to if we'll be able to fine tune the multimodality of GPT-4 as well. Literally the first sentence in the article: > Fine-tuning for GPT-3.5 Turbo is now available, with fine-tuning for GPT-4 coming this fall.
- brianjking 3y agoYes, nowhere does it say anything about MULTIMODAL fine tuning being supported/unsupported when GPT-4 fine tuning becomes available. This was my entire point. I did read the article.
- jron 3y agoAre there major advantages of GPT-3.5 Turbo tuning over PEFT/LoRA with Llama2?
- minimaxir 3y agoLatency and cost. GPT-3.5-Turbo is very very fast (for reasons I still don't understand) and cost is very very low even with the finetuning premium. Llama2 is still slow even with all the LLM inference tricks in the book and you need to pay for expensive GPUs to get it to a production-worthy latency, along with a scaling infra if there is a spike in usage.
- eldenring 3y agoGPT-3.5 is much, much smarter than Llama2. Its not nearly as close as the benchmarks make it seem.
- intellectronica 3y agoIndeed, and this is really missing from the public discourse. People are talking about Llama 70b as if it was a drop-in replacement for gpt-3.5, but you only have to play with both for half an hour to figure out that's not generally the case and only looks true in cherry-picked examples.
- Tostino 3y agoSo, as somebody who has fine tuned llama2 (13b) on both a new prompt template / chat format, as well as instruction following, summarization, knowledge graph creation, traversing a knowledge graph for information, describing relationships in the knowledge graph, etc. It is able to use the knowledge graph to write coherent text that is well structured, lengthy, and follows the connections outlined in the graph to the logical conclusions, while deriving non-explicit insights from the graph in it's writings. Just to say, i've seen a giant improvement in performance from Llama2 by fine tuning. And like I said, just 13b...I am perfecting the dataset with 13b before moving to 70b. 3.5-turbo is sometimes okay, i've tested it moderately for the same tasks i've been training/testing Llama2 on, and it's just a bit behind. Honestly, my fine tune is more consistent than gpt4 for a good number of the tasks i've trained.
- deleted 3y ago[deleted]
- tuckerconnelly 3y agoQuestion about privacy: OpenAI doesn't use API calls to train their models. But do they or Microsoft still store the text? If so, for how long? Overall, I think this is great, and can't wait for the 16k fine-tuning.
- haldujai 3y agoNot sure about direct OpenAI API calls but with the Azure offering they store prompts and output for 30 days to monitor for abuse. There is an application form if one wants to be exempted from this requirement. https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy#how-does-the-azure-openai-service-process-data https://learn.microsoft.com/en-us/legal/cognitive-services/o...
- 3abiton 3y agoDoes the finetuned model reside on OpenAI's servers? If so, what privacy guarantees that openai won't utilize it later for expanding gpt5?
- flangola7 3y agoInsist on such guarantees in the contact.
- jakeduth 3y agoYes they are stored on OpenAI's servers. The API calls are not used for model training per the TOS. However, not that I'm accusing OpenAI of anything, but there's no way to independently validate this. But their guarantee is clear for the API (the ChatGPT web app is different, but you can disable training if you give up the history feature). > At OpenAI, protecting user data is fundamental to our mission. We do not train our models on inputs and outputs through our API. > ... > We do not train on any user data or metadata submitted through any of our APIs, unless you as a user explicitly opt in. > ... > Models deployed to the API are statically versioned: they are not retrained or updated in real-time with API requests. > Your API inputs and outputs do not become part of the training data unless you explicitly opt in. - https://openai.com/api-data-privacy https://openai.com/api-data-privacy
- Karrot_Kream 3y agoGiven that they're offering davinci-002 for completion, I was wondering whether they went back to offering an "unsafe" model, but both davinci-002 and babbage-002 enter an infinite loop for any "unsafe" completion where text-davinci-003 and text-curie-001 is fine. Sigh, I guess OpenAI really does not want to offer unsafe models for usage.
- freeqaz 3y agoWhat is an "unsafe" model?
- Karrot_Kream 3y agoA model which outputs things that OpenAI deems is unsafe. Try getting text-davinci-003 to complete instructions about building Molotov cocktails and compare that with davinci-002.
- shagie 3y agocurl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "text-davinci-003", "prompt": "Describe the steps for creating a Molotov cocktail.\n\n1.", "temperature": 1, "max_tokens": 256, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' This worked without any issue at all and gave a satisfactory answer. You are expected to run this through the moderation endpoint ( https://platform.openai.com/docs/guides/moderation https://platform.openai.com/docs/guides/moderation ) to put the guardrails on and avoid unsafe content from getting to an end user. Incidentally, that prompt doesn't appear to tickle the moderation endpoint into flagging it though the violence score was the highest. { "id": "{redacted}", "model": "text-moderation-005", "results": [ { "flagged": false, "categories": { "sexual": false, "hate": false, "harassment": false, "self-harm": false, "sexual/minors": false, "hate/threatening": false, "violence/graphic": false, "self-harm/intent": false, "self-harm/instructions": false, "harassment/threatening": false, "violence": false }, "category_scores": { ... "violence": 3.33226e-05 } } ] } Running it with the resulting text from that was generated by text-davinci-003 didn't get flagged either, though the score for violence went up to '"violence": 0.01034669'.
- monkeydust 3y agoWere using RAG with some prompt engineering and guardrails with Nvidia Neom, would fine tunning help further with hallucinations and quality of responses? Any research specifically on this?
- thumbsup-_- 3y agoCan anyone share their thoughts on how does this compare to fine-tuning Llama2 13B or 70B which usually requires renting GPUs from one of the cloud providers? In case of openAI, the infra doesn't needs to be rented but the cost is covered in usage pricing but want to hear from someone who has experience fine-tuning Llama2 models that how it compares.
- SparkyMcUnicorn 3y agoI haven't fine-tuned GPT-3.5-turbo yet, but overall llama2 is probably going to be cheaper. Especially if 13B is good enough for you. You can get roughly $0.003/1K tokens using modal.com for inference on a fine-tuned llama2 13B model, and there's definitely cheaper options available. Unless you have a ton of data, the cost for training llama2 is usually a few dollars.
- swyx 3y ago> the cost for training llama2 is usually a few dollars. nit: thats the cost for finetuning, not pretraining
- SparkyMcUnicorn 3y agoYeah, definitely not pretraining. I've been referring to fine-tuning as training, since that's what's happening when you "fine-tune" a llama model.
- easygenes 3y agoI'm curious about this. Can you point me to, e.g. some example code for setting up an inference endpoint with a base llama2 model on modal.com?
- SparkyMcUnicorn 3y agoHere's one if their tutorials using vLLM, and they have a few other guides and example repos as well. https://modal.com/docs/guide/ex/vllm_inference https://modal.com/docs/guide/ex/vllm_inference https://github.com/modal-labs https://github.com/modal-labs Alternatively, Runpod is fairly cheap and easy to get stuff running in a few minutes and can be point/click only using their templates. https://www.runpod.io/console/gpu-secure-cloud?template=f1pf20op0z https://www.runpod.io/console/gpu-secure-cloud?template=f1pf... ("serverless" example) https://github.com/ashleykleynhans/runpod-worker-oobabooga https://github.com/ashleykleynhans/runpod-worker-oobabooga
- ilaksh 3y agoThis is amazing. Slightly related, anyone know when Code Llama is coming out?
- bennyschmidt 3y agoGPT 3.5 is so bad it's useless to me - for writing it's too repetitive of the same kind of jargon, for coding it's wrong way too often. The NLP is also worse, I have to be more explicit. It's just an average chat bot IMO. GPT 4 @ $20/mo. is significantly better at everything, I use it for doing stuff in Angular lol - when you have an AI explaining the why behind everything, this over-engineered mess of a framework starts to actually make sense. Definitely nice to have around as a translator/teacher or troubleshooting assistant. Can't imagine googling for answers to problems if this gets any better. The main thing is just habit - GPT 4 is lower effort to arrive at more direct, bespoke answers. The one feature I want is built-in prompt-splitting, so we don't have to use third-party tools. In my all-wise random person's opinion: Forget the old versions of GPT, and forget the phony ethics, and focus on the best version of this technology, sell it for $20/month, make billions and disrupt a lot of things online.
- menzoic 3y agoUse code interpreter to upload your files and prompt it to ask you a serires of questions to know what to do next
- grubbyboy 3y agoCode interpreter is quite good. I used it to create graphs, convert csv to JSON, write a complex Bash script, and regex. It's impressive.
- weird-eye-issue 3y agoUploading a file to Code Interpreter does not magically increase the prompt context length. It will just read in part of the file or write code that operates on the file, depending on your prompt
- sheepscreek 3y ago> forget the phony ethics, and focus on the best version of this technology I’ve experimented a lot between the censored and uncensored versions of Llama 2. Based on this, I’ve concluded that fine-tuning for political correctness and ethics negatively affects all answers. They become repetitive and washed out.
- albert_e 3y agoJudging by their code example ... one user can have only a single fine tuned model? I am speaking as an individual developer - nor an enterprise. But would ne hood to know answer to both types of accounts.
- tedsanders 3y agoYou can have many.
- steno132 3y agoAnd OpenAI continues to ship. As promised, they released GPT3.5 fine-tuning today. They opened GPT4 API access a few months ago. In a few months, they'll release GPT4 fine-tuning. Many favor open source AI, and criticize OpenAI for not being open enough. But the most important thing is, OpenAI innovates. Fast. Llama, Bard, FB's open source stuff is good but it's lightyears behind OpenAI. You have to credit them for that.
- syntaxing 3y agoPrior to LLaMA 2, I would have agreed with you but LLaMA 2 is a game changer. The 70B performance is probably between 3.5 and 4. But running it personally isn't cheap. The cheapest I found is about $4/hr to run the whole thing. I only spend around $3 on average a month on GPT-3.5 API for my personal stuff.
- armini 3y agoHow are you currently hosting your LLaMA 2? Any tips, tricks or advice?
- syntaxing 3y agoIt depends on your needs. For instance, do you want to host an API or do you want to have a front end like chatGPT? Chances are, text-generation-webui [1] should get you pretty close to hosting it yourself. You simply clone the repo, download the model from huggingface using the included helper (download-model.py) and fire up the server with server.py. You can connect to it by SSH port tunneling on port 7860 (there's other way like Ngrok but SSH tunneling is the easiest and secure). As for hosting, I found that runpod [2] has been the cheapest (not affiliated, just a user). All the other services tend to add up more than them when you include bandwidth and storage. There's some tutorials online [3] but a lot of them use the quantized version. You should be able to fit the original 70B with "load_in_8bit" on one A100 80GB. [1] https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui [2] https://www.runpod.io/ https://www.runpod.io/ [3] https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oobaboogas-text-generation-webui/ https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oo...
- osigurdson 3y agoIf you are using C#, I think my take on fine tuning is far more intuitive than just using the raw APIs. See https://www.nuget.org/packages/OpenAILib https://www.nuget.org/packages/OpenAILib. Now, just gotta hunker down and implement the new ChatGPT fine tune feature.
- tiku 3y agoMy calls suddenly get emoji's in the return texts. Fun times but now what my client wants haha.
- weird-eye-issue 3y agoYou should probably be using the static model shouldn't you?
- Yaappii 3y agogoddamn it i hate ai
- lynx23 3y agoSafety standards. One might call it "our censorship model" but that would be too close to the truth, so call it safety standards...
- speedgoose 3y agoDo people really wish for OpenAI models to output healthcare advices without disclaimers, plans to end humanity, child porn fictions, torture instruction books, and similar? The minute it happens without complicated bypasses, the society would say stop to generative ai, and rightfully so. Many people already got spoked when they tricked ChatGPT to say/repeat scary things.
- junofan 3y agoYou can get all of that already by typing 4chan.org. Society seems unaffected.
- lynx23 3y agoYou forgot to list the most hypocritical--american--topic filtered, erotic content. I know, the american world will go down in flames if a generative AI ever outputs something sexy. You can google all these scary things these days already. And prior to that, you could go to a bookstore and find most of what you mentioned. Or go to asstr.org for your fucked up sex stories Pretending a content filter on a generative AI would make anything better is simply bigottery.
- speedgoose 3y agoI’m saying that no content filter would make things worse because it would make the technology not acceptable.
- victor106 3y ago> For example, a gpt-3.5-turbo fine-tuning job with a training file of 100,000 tokens that is trained for 3 epochs would have an expected cost of $2.40. How do you calculate the number of tokens required?
- weird-eye-issue 3y agoConvert the text to tokens and get the length?
- facia 3y ago[dead]
- k3ntar0 3y agoThe cost is very low for fine-tuning, so it seems that PEFT, which is clearly different from regular FT or RoLA or something, is being used. What do you think is being used?
- bambax 3y agoCould someone explain in simple terms exactly what fine-tuning does? Does it show the model how to answer questions, or does it give it new information, or both? Is there a way to restrict answers to the fine-tuned data? For example, if one would want to use an LLM to answer questions regarding a large, private knowledge base, would it make sense to fine-tune a model on this knowledge base? If yes, how does one reduce hallucination? And would that perform better than feeding possible source documents as part of the prompt every time?
- skerit 3y ago> For example, if one would want to use an LLM to answer questions regarding a large, private knowledge base, would it make sense to fine-tune a model on this knowledge base? I initially also thought this would be one of the best use cases for fine-tuning (teaching the model new data), but I've seen quite a few people say fine-tuning should not be used to teach the model new data, but more like new formatting and style of response. This blog post seems to concur. I do wonder how OpenAI does fine-tuning. I'm guessing it doesn't use Lora.
- textcortex 3y agoMost probably they are doing something called prompt tuning. This creates a small ai model that adds virtual tokens to prompt before passing to original model: https://developer.nvidia.com/blog/an-introduction-to-large-language-models-prompt-engineering-and-p-tuning/ https://developer.nvidia.com/blog/an-introduction-to-large-l...
- tedsanders 3y ago>Could someone explain in simple terms exactly what fine-tuning does? Fine-tuning shows the model examples of sequences it should produce. The model is updated to become more likely to produce sequences like those examples. What precisely 'like those examples' means for brand new prompts unlike those in the training distribution is the black magic of generalization. >Does it show the model how to answer questions, or does it give it new information, or both? It can be used to teach style, or information, or both. >Is there a way to restrict answers to the fine-tuned data? There is no foolproof way to restrict answers to fine-tuned data. You might be able to approach decent performance if you show it examples of refusing on all topics not related to X. >For example, if one would want to use an LLM to answer questions regarding a large, private knowledge base, would it make sense to fine-tune a model on this knowledge base? Short answer: I wouldn't recommend fine-tuning. Long answer: it depends on your task, your expertise, and your tolerance for collecting large datasets and iterating. I generally recommend retrieval. Putting info in the input has a few advantages over fine-tuning: you can check where information is coming from, and it's easier for the model to answer without hallucinating (akin to a student taking a test with open notes they can refer to, rather than trying to remember a textbook they read a week ago). Retrieval is best at lookup type questions and is worse at questions requiring comparisons or mixing of many pieces of source data; possible fine-tuning has some edge there.
- chaosbolt 3y ago>release what was supposed to be open as a free beta to fix bugs >release its more powerful brother as a subscription nased service >heavily nerf both >release fine tuning to maybe make the nerfed gpt 3.5 as good as it was at launch but only if you finetune it well enough >keep the unnerfed version for internal use at microsoft >profit I mean at least Google is honest about it, they have the best product, you won't get it because it's more valuable as an internal tool than public, sure announce Bard after gpt launches to not have your stocks go down but it's bad and even then will probably never launch. At least meta made their nerfed version opensource. I legit was a beast with the gpt 4 of a couple months ago, now I'm back to a 1 man developer, using it now makes me waste time more than gain it, since I have to fix its errors, might as well do it myself... so I can see how you don't want to give it to others. If your reply is going to be something obviously wrong like "it wasn't nerfed" then just don't waste your time man...
- yakorevivan 3y ago[dead]