I think people tend to just not understand what autoregressive methods are capable of doing generally (i.e., basically anything an alternative method can do), a

I think people tend to just not understand what autoregressive methods are capable of doing generally (i.e., basically anything an alternative method can do), and worse they sort of mentally view it as equivalent to a context length of 1.