I used turned on heroku maintenance mode for a couple of apps I didn't care to patch right away, according to its description it prevents requests from reaching

I used turned on heroku maintenance mode for a couple of apps I didn't care to patch right away, according to its description it prevents requests from reaching dynos so should be good for preventing access to existing data/stored keys in the meantime.