Let me share a sanitized C++ memory corruption story. We were testing our application the night before a big demo for our client. Ours was a C++ OpenGL applic

Let me share a sanitized C++ memory corruption story. We were testing our application the night before a big demo for our client. Ours was a C++ OpenGL application with several "phases" the user could enter from a "main menu" type screen. Our demo explored these phases in a pre-planned order. All went well, until somebody decided to try doing the demo in a slightly different order. Once the tester visited a certain phase of the application, the thing crashed and burned. The stack trace was littered with some incomprehensible container-implementation-detail junk. Debuggers weren't pointing at an obvious sign, other than that something was dereferencing a garbage pointer. We noticed that if we went back to the original canned order of operations (this was a demo, after all), the application didn't crash. But this still made us nervous. We found that we could sometimes swap a few phases early in the application's run with no ill effect. Here's part of the class responsible for some of the higher layers of the stack trace, from one of the lower level libraries we build all of our applications on (this is important because, for hysterical raisins, we can't source-step debug those libraries in production). Can you spot the bug? #include <map> #include <string> #include <vector> struct Message; class MessageCentral { public: void HandleMessages() { // messages were collected somewhere and organized by their queue. const std::vector<std::string> queues_with_new_messages; for (const std::string &queue : queues_with_new_messages) { const std::vector<Message> msgs; // = messages_for_queue(queue); const auto &handlers = handlers_[queue]; for (auto &h : handlers) { for (const auto &msg : msgs) { h(msg); } } } } void RegisterMessageHandler(const std::string &queue_name, std::function<void(Message)> handler) { handlers_[queue_name].push_back(handler); } private: std::map<std::string, std::vector<std::function<void(Message)>>> handlers_; }; (Note: this is probably way easier to figure out with all of the other code needed to implement MessageCentral taken out! It took us three engineers working for about 2.5-3 hours each, with lots and lots of gdb and printf debugging, to nail this one down. This particular crash would have been impossible in Rust, because the compiler would have rejected the Rust equivalent of the above C++ code.) (oh, by the way, we couldn't use runtime memory sanitizer tools like ASan or valgrind because the application linked to CEF. In retrospect, we could have dropped in a fake CEF that didn't interfere with ASan or valgrind, but that would have required build resources we didn't have available in the frenzied night before a big demo.)