> I just don't want to spin up EC2 instances manually, get the connections all working, make sure I can reset state, etc. I've been thinking about this lately.

> I just don't want to spin up EC2 instances manually, get the connections all working, make sure I can reset state, etc. I've been thinking about this lately. I wonder if we could standardize a benchmark format so that you could automatically do the steps of downloading the code, setting up a container (on your computer or in the cloud), running the benchmarks, producing an output file, and making a PR with the output. So developers would go "here's my benchmark suite, but I've only tested it on my machine", and users would call "cargo bench --submit-results-in-pr" or whatever, and thus the benchmark would quickly get more samples. (With graphs being auto-generated as more samples come in, based on some config files plus the bench samples)