3 ms·
The balance may tip in favour of fine tuning once we have made small breakthroughs in this space. It might be especially useful in enterprise contexts where you
by simianwords 1y ago
The balance may tip in favour of fine tuning once we have made small breakthroughs in this space. It might be especially useful in enterprise contexts where you can have one model per company trained through all Wiki, code, documentation etc.
- simonw 1y agoThat right there is the thing I'm most skeptical of. It's so very obviously what every company wants: a custom model fine-tuned on their internal documentation and code. And yet stories of it actually working are incredibly rare! The closest I've heard to a success story in that space is Jane's Street who fine-tuned their own model because they use OCaml more than anyone else: https://www.youtube.com/watch?v=0ML7ZLMdcl4 https://www.youtube.com/watch?v=0ML7ZLMdcl4 I am confident that any startup today who could provably demonstrate that "we can fine tune a model on your company's internal code and documentation and have it answer questions about them" would have enormous financial success. I'll believe it works when I see that!