4 ms·
I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close eas
by antupis 4y ago
I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.
- bpodgursky 4y agoNone of the GPT models rely on annotated or classified data. It's unsupervised.
- IanCal 4y agoThe final training steps of gpt 3.5 did, and we assume the same for 4. The RLHF step.
- whimsicalism 4y agoI would say semi-supervised, RLHF is not quite supervised learning. None of this data is from Reddit, Twitter, etc.
- wordpad25 4y agoIt’s too easy to scrape them, but even that is already done by many intermediary companies all too happy to resell that data.
- antupis 4y agoYup but you still need up to date data and now companies are realising what kinda gold mine random forum posts about nightlife in Glasgow are they will start monetising it and probably try at least kill middle market.