3 ms·
> In practice, this would be an enormous number of states The paper mentions this explicitly: For GPT-3 (Brown et al., 2020), this represents 5 × 10^11 traini
by magicalhippo 2y ago
> In practice, this would be an enormous number of states
The paper mentions this explicitly:
For GPT-3 (Brown et al., 2020), this represents 5 × 10^11 training tokens, which pales in comparison with the number of non-zero elements in Qf, given by T^K+1 ≈ 10^9632.