Wrote a stored procedure that joined with our `sessions` table. Worked fine in local dev and staging of course. And it was fine in prod too for 6 weeks or so be

Wrote a stored procedure that joined with our `sessions` table. Worked fine in local dev and staging of course. And it was fine in prod too for 6 weeks or so because the other end of the join didn't have any rows yet. Eventually some changes landed so the other table starts receiving data and the cost of the join begins to increase. Some time around midnight on a Friday night, performance degrades sufficiently for queries to start timing out. No-one can read a session, the site is effectively down for everyone (30m DAU). Ops team lost their whole weekend tracing back to what I'd broken, because the delay since deployment confused the issue so much.