3 ms·
on top of the hardware requirements ($10s of thousands of GPUs are needed for something a language model as big as gpt-3) there's also a lot of work involved in
by turmeric_root 4y ago
on top of the hardware requirements ($10s of thousands of GPUs are needed for something a language model as big as gpt-3) there's also a lot of work involved in RLHF models like chatgpt. you need to pay people to write and review thousands/tens of thousands of responses for training. see 'methods' here: https://openai.com/blog/chatgpt/ https://openai.com/blog/chatgpt/
- coolspot 4y agoLAION’s Open Assistant trying to address that with gamified crowdsourcing. https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant