2 ms·
It's really difficult to dig up information from that website. What is its architecture? How many parameters? What is the tokenizer and what size? What is the m
by jxy 3y ago
It's really difficult to dig up information from that website. What is its architecture? How many parameters? What is the tokenizer and what size? What is the max context length? How many tokens used for pre training? How many tokens used for fine tuning?
For performance, what is mT0x they are comparing against? mt0-xl? mt0-xxl? mt0-xxl-mt? Anyway, if it's any of these mt0-*, it's not really useful in practice.
- LoganDark 3y agoThe model weights seem to be available here, as well as some technical details: https://huggingface.co/CohereForAI/aya-101 https://huggingface.co/CohereForAI/aya-101
- jxy 3y agoYeah. Their arxiv has more: https://arxiv.org/pdf/2402.07827.pdf https://arxiv.org/pdf/2402.07827.pdf Specifically, > Aya is built by fine-tuning 13B parameter mT5 model There is no mention of the base model any where in that website.