Note that this behavior changed with the new force-quit gestures. When you quit from the app switcher it was decided that the user explicitly wanted to reset UI

Note that this behavior changed with the new force-quit gestures. When you quit from the app switcher it was decided that the user explicitly wanted to reset UI state, so the UI state restoration data gets wiped. So that behavior you're describing for Photos is expected.