I don't know how well it would work, but I think you could still perform alignment using a single column assuming that you would expect to see patches with the

I don't know how well it would work, but I think you could still perform alignment using a single column assuming that you would expect to see patches with the same color connected together more often than not. Especially if you're trying to do video since then you can also exploit temporal coherency (meaning that you expect many objects to be stable over time). I'm sure in my head it's much simpler than practice, but it seems like it is achievable at least in theory. Sounds like a very fun problem for computer vision researcher to work on. Probably not terribly difficult to generate a bunch of test data from existing 360 cameras by downsampling.