3 ms·
Any idea why everyone seems to be using 8 experts? (Or was GPT-4 using 16?) Did we just try different numbers and found 8 was the optimum?
by andai 3y ago
Any idea why everyone seems to be using 8 experts? (Or was GPT-4 using 16?) Did we just try different numbers and found 8 was the optimum?
- wongarsu 3y agoProbably because 8 GPUs is a common setup, and with 8 experts you can put each expert on a different GPU