7 ms·
My own experiments made me think that the impact of finetuning is comparable to that of a molecule in a drop in a bucket. > “AskHN” is a GPT-3 bot I trained on
by btbuildem 4y ago
My own experiments made me think that the impact of finetuning is comparable to that of a molecule in a drop in a bucket.
> “AskHN” is a GPT-3 bot I trained on a corpus of over 6.5 million Hacker News comments to represent the collective wisdom of the HN community in a single bot.
I'm assuming you used the openai fine-tuning pathway to make a custom model?
Have you tested the responses on vanilla GPT3 vs your custom model?
I'd be curious to see the comparison.
- danuker 4y agoYeah, to me it looks like the learning rate was way too low to make a difference. I don't see any of the sublime and succinct snark.
- notahacker 4y agoYeah. Also full of GPT-3isms like "ultimately the choice ... comes down to the specific project and its ... requirements" and not nearly contrarian enough A bot focused on the output of HNers would insist on providing arguments against going through Google's interview process in the first place and suggestions that the correct answer to "Python or R" should be Haskell or Julia and would never suggest prioritising emotional vulnerability or being a happy person!
- danuker 4y agoThank you for the laffs =)
- clark-kent 4y agoFrom the article, they did not use fine-tuning. This is semantic search + GPT-3 to provide human-like answers.
- btbuildem 4y agoThanks! I missed that part. The semantic search approach seems to focus the answers better than fine-tuning; at the cost of preloading the prompt with a lot of tokens, but with the benefit of a more constrained response.