3 ms·
Of course the implementation isn’t the issue. It’s the training data and the compute machines. Open source is pretty meaningless here
by jonathankoren 4y ago
Of course the implementation isn’t the issue. It’s the training data and the compute machines.
Open source is pretty meaningless here
- jfoster 4y agoIn fact, this kind of reverses things, doesn't it? Open source is built on the assumption that you can do more with source code than with binaries. In the case of AI models, the computed weights of models are what's valuable, and the source code used to achieve them is less useful.
- aero-glide2 4y agoHow much would it cost in training to match dall-e 2?
- abrichr 4y agohttps://twitter.com/alexjc/status/1347458546636619778?s=21&t=xJ3ftcaqJm-0sZcu_WsPKg https://twitter.com/alexjc/status/1347458546636619778?s=21&t... > The blog post says 256 GPUs for 2 weeks, so: > DALL·E would cost $131,604 to train on AWS, assuming a p3.16x-large at market rates. Could be as low as $40k if you already paid for reserved instances.
- emadm 4y agoImages actually used our open LAION 400m dataset plus 400m of their own. On compute there is more than enough compute available to open source now via LAION and Eleuther AI to train these models, will just a bit of time.