5 ms·
I would be very curious to see your notebook - while this simple approach works well with GPT, we are not getting the results we'd want with a more complex ques
by MasterScrat 7y ago
I would be very curious to see your notebook - while this simple approach works well with GPT, we are not getting the results we'd want with a more complex question/answer model that uses GPT2. So I'd love to see your implementation details!
> Having said that you seem to train here on very little.
The datasets provided in the notebook are really meant to be fallbacks for people who are not willing to use their own chat log data. When training on my own data, I have about 500k messages, which starts being enough to get interesting results.
edit: wow, I see you're training on "14M facebook messages", that's impressive - do you actually chat that much?!
- Tenoke 7y agoI just pushed it, the blog post (which includes the notebook) is here[0]. It's 14mb of data, not sure how many messages it actually comes down to but FB Messenger has been my main platform for talking to friends for the last decade. 0. https://svilentodorov.xyz/blog/gpt-15b-chat-finetune/ https://svilentodorov.xyz/blog/gpt-15b-chat-finetune/
- MasterScrat 7y agoAwesome :D We've published the third notebook: https://colab.research.google.com/drive/1XYNef9zcHhTjt6kM6ydL9oXTshoRknIV https://colab.research.google.com/drive/1XYNef9zcHhTjt6kM6yd... We would gladly have your feedback on our approach