4 ms·

Is it possible to have a test stage that replays/simulates the entire previous 24 hours of production to compare previous output with new output. This might let

Is it possible to have a test stage that replays/simulates the entire previous 24 hours of production to compare previous output with new output. This might let you know if a fix to a known bug actually makes things worse. Obviously I'm not inferring this is a step mozilla should already have in place.

woliveirajr 9y ago
That's a huge step. Lots of data to save for the input, lots of diff output, and you have to look for all details of the output, since many of them are desirable. Then you would automate the output processing, and a wrong rule would miss the new bug (because it was unexpected, so the automation would account for it).