4 ms·
Some acknowledgements! - Cedille is based on GPT-J, a model developed by the grassroot EleutherAI team: https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/
by MasterScrat 5y ago
Some acknowledgements!
- Cedille is based on GPT-J, a model developed by the grassroot EleutherAI team: https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/ https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/
- Cedille was also generously supported by the Google TRC program: https://sites.research.google/trc/about/ https://sites.research.google/trc/about/
- jgalt 5y agoHow much compute would you need to build a GPT3 like sized model in terms of number of GPUs (A100) and storage? Not considering the money aspect here.
- matbilodeau 5y agoAside from creativity, is it much harder to compute french or other latin based languages compared to english? I'm not a linguist but just with conjugations the ruleset seems simpler with the latter. How does that reflect on the development of cedille ?