There's been a huge amount of work on image transformers since the original VIT. A lot of it has explored different schemes to slice up the image in tokens, and

There's been a huge amount of work on image transformers since the original VIT. A lot of it has explored different schemes to slice up the image in tokens, and I've definitely seen some of it using a multiresolution pyramid. Not sure about the RL part - after all, the higher/low-res levels of the pyramid would add less tokens than the base/high-res level, so it doesn't seem that necessary. But given the sheer volume of work out there I can bet someone has explored this idea or something pretty close to it already.