3 ms·
You download the 2.76TB of data. Then you run it through Llama's training script for a couple months on 40 NVidia A100's, and you should have yourself a pretty
by tinco 4y ago
You download the 2.76TB of data. Then you run it through Llama's training script for a couple months on 40 NVidia A100's, and you should have yourself a pretty fine large language model you could use to host your own ChatGPT service. It'll be significantly worse than ChatGPT for reason's that aren't yet fully clear because OpenAI switched its mission from protecting the earth from nefarious AI developments, to being itself being the origin of possibly nefarious AI developments.
- DigitalDopamine 4y agoRenting 40 nvidia a100s is around 70k dolar per month (on vultr i see). So this would only cost 420k for 6 months. Seems doable. Is 40 a100s enough though? I am interested in what this would cost.
- mlboss 4y agoIt would be great if this can be done on 3090s. Used 3090 usually costs $500-1000 to buy.
- nl 4y agoLLama 65B used 2048 80G A100s for 21 days: > When training a 65B-parameter model, our code processes around 380 tokens/sec/GPU on 2048 A100 GPU with 80GB of RAM.[1] Note that you probably need to budget for double to triple that because things go wrong and it usually takes multiple starts to get a good training run. Smaller models are cheaper though. [1] https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf
- DesiLurker 4y agoit pains me to see AMD just sitting on their asses through this incredible development of AI & possibly AGI. if they still cant get their shit together then they should spin-off the discrete gpu division into something purely compute focused. I believe now there is enough momentum in the AI/ML space to fully develop innovative ideas on h/w front.