3 ms·
Theres never going to be quality data at tge scale they need it. At best, theres a slow march to incremental improvements that look exactly like how human cult
by cyanydeez 2y ago
Theres never going to be quality data at tge scale they need it.
At best, theres a slow march to incremental improvements that look exactly like how human culture developed knowledge.
And all the downsides will remain, the same way people, despite hundreds of good sourxes of info still prefer garbage.
- jerpint 2y agoVideo data is still very much untapped and likely to unlock a step function worth of data. Current image-language models are trained mostly on {image, caption} pairs with a bit of extra fine tuning
- KPGv2 2y agoDo you think it matters that there's orders of magnitude less video (and audio) data than text data?
- jerpint 2y agoI’m not sure I agree, text is full of compressed information but lacks all of the visual cues we all use to navigate and understand our world. Video data also has temporal components which text is really bad at.
- mike_hearn 2y agoOpenAI has been training on YouTube for years.
- luma 2y agoNobody close the the edge of this tech seems to believe that this is true, and the 4o release suggests synthetic approaches work well.
- cyanydeez 2y agoBeliefs and outcomes are different things.