3 ms·

I wonder if there might already be enough datasets laying around that even if we stopped making new ones, models can generate entire new datasets on their own,

I wonder if there might already be enough datasets laying around that even if we stopped making new ones, models can generate entire new datasets on their own, process/label them on their own, and generate new and improved models in a closed feedback loop. The only data that's really needed going forward is information on current events.

ehnto 4y ago
There is no mechanism to fact check generated data, it could be slowly making a fantasy world for itself. There's already no factuality to ChatGPT, it has no understanding of if the things it says are based in fact. Just an example, you ask it when a certain variety of grape was introduced in Australia and it just picks a probable Australian town name. Different each time. It doesn't know how to be right in that circumstance.