5 ms·
No, tokenization is not the only reason. A next-word predictor has fundamentally a hard time executing algorithms, even as simple as counting.
by teiferer 6mo ago
No, tokenization is not the only reason. A next-word predictor has fundamentally a hard time executing algorithms, even as simple as counting.
- mitthrowaway2 6mo agoCounting is one of the algorithms that can be expressed by a RASP program, which transformers closely approximate.
- MarkusQ 5mo agoClose famously counts in horseshoes and hand grenades. Algorithms, just as famously, are a domain where off-by-one is still wrong.