That’s a great question - typically memory in transformers scale as O(N^2) with token count, so there must be an upper limit, but I would bet it to be far more

That’s a great question - typically memory in transformers scale as O(N^2) with token count, so there must be an upper limit, but I would bet it to be far more than 4096 tokens