2 ms·
It sounds like that's a good use for encoder-decoder architecture. GPT-3 and LLaMA are decoder-only architectures. Training that encoder sounds hard, though.
by pclmulqdq 4y ago
It sounds like that's a good use for encoder-decoder architecture. GPT-3 and LLaMA are decoder-only architectures.
Training that encoder sounds hard, though.