3 ms·

BF involves a lot of repeated symbols, which is hard for tokenized models. Same problem as r's in strawberry.

• bwestergard 7mo ago
  Interesting. So why do the models seem to handle deeply nested Lisp expressions just fine?
  • kgeist 7mo ago
    Probably because there's a ton of code that deals with nested parentheses across languages in the training data, and models have learned how to work around tokenization limitations, when it comes to parentheses.
    • astrange 7mo ago
      It's because the models wouldn't work for coding if they couldn't do nested scopes, so people don't release models unless they work. They can only do it in a limited form though, because transformer models only have limited "memory". I don't think they can fully implement parsing.