3 ms·
There is nearly an unlimited amount of training data. As of so far models have been eating up text. We still have sound, image, video, temperature, gravimetrics
by pixl97 4y ago
There is nearly an unlimited amount of training data. As of so far models have been eating up text. We still have sound, image, video, temperature, gravimetrics, and other sources that we can feed multimodal models. And that's not even including training models from learning from the world itself.
- anonzzzies 4y agoSound, image and video we’ll ravish through (already happening); is temp and gravimetrics valuable? (I don’t know; it’s a question) Games might be a source too. But yes, the world itself is a good source. So maybe that won’t freeze it over; computing power/energy?
- pixl97 4y ago>computing power/energy? Energy is "probably" not a big deal. If you're looking at long term oversupply from green energy sources, it's probably not hard to train with bursty, but very cheap power like this. Compute is currently the biggest limitation, and will remain so far a long time, as long as scaling continues.
- akiselev 4y ago> gravimetrics Calling it now: humanity will seal its fate the day we hook up LIGO to ChatGPT and it gets corrupted by gravitational waves from the beginning of time when the Great Old Ones freely roamed the universe.