4 ms·
Even in the case of a full size GPT, one could equivalently write out a lookup table for every possible permutation of tokens in the context window and the resu
by Imnimo 4y ago
Even in the case of a full size GPT, one could equivalently write out a lookup table for every possible permutation of tokens in the context window and the resulting output distribution. The table would of course be impossibly large.
- oezi 4y agoDue to combinatorial explosion this would be infeasible even for a single input vector.