3 ms·
Interesting. Thanks for the comment. What if you ask it to add numbers with more digits?
by jjtheblunt 3y ago
Interesting. Thanks for the comment. What if you ask it to add numbers with more digits?
- GaggiX 3y agoThe model is trained with a fixed number of tokens, I don't remember if the models I trained have sinusoidal embeddings or learnable positional embeddings, in the latter there would be no embedding to encode the position, in the former I think it would cause problems with the sinusoidal embedding layer as the sine and cosine would wrap around.