4 ms·
I'm not sure hardware is the bottleneck here. I've had "AI" regurgitate reddit and 4chan threads verbatim to me. What happens when this mine of free data runs
by otabdeveloper4 2y ago
I'm not sure hardware is the bottleneck here.
I've had "AI" regurgitate reddit and 4chan threads verbatim to me. What happens when this mine of free data runs out? (Basically it already has, in no small part due to the effects of "AI" itself.) Will we need to actually pay creators for content? Seems economically unfeasible, the cost of content creator + AI is more than just hiring the content creator directly.
- llm_trw 2y agoHardware is very much the limit. The more compute you have to train your models the better the models become. I've been working on a family of (new) deep learning models which outperform everything else that's been published, at between x1e3 to 1e9 the compute on the same data. If I had the GPT4 training cluster I'd be able to run it on cifar10 without having to make sacrifices, yes 32x32 color images solved with a trillion parameters.