3 ms·
11B parameters, so I guess it's FLAN-T5[1]? combined with retrieval transformer (something like RETRO[2])? 1. https://arxiv.org/abs/2210.11416 https://arxiv.or
by lossolo 4y ago
11B parameters, so I guess it's FLAN-T5[1]? combined with retrieval transformer (something like RETRO[2])?
1. https://arxiv.org/abs/2210.11416 https://arxiv.org/abs/2210.11416
2. https://arxiv.org/abs/2112.04426 https://arxiv.org/abs/2112.04426