5 ms·
"Based on all possible outcomes of pending changes, SubmitQueue constructs, and continuously updates a speculation graph that uses a probabilistic model, powere
by huac 8y ago
"Based on all possible outcomes of pending changes, SubmitQueue constructs, and continuously updates a speculation graph that uses a probabilistic model, powered by logistic regression. The speculation graph allows SubmitQueue to select builds that are most likely to succeed, and speculatively execute them in parallel"
This is either brilliant or just something built for a promotion packet
- mcqueenjordan 8y agoPromotion-oriented design, no doubt.
- burakcosk 8y agono it is not, this whole thing being used in production and really reduces the time changes submitted and they are merged into master. and the master is almost always green, meaning developers can build and test any piece of code without problem. They have designed this as a result of a need, not just a fancy project.
- draw_down 8y ago"I don't experience this problem, therefore this problem is not real."
- sundargates 8y agoI can guarantee you that none of the ideas on the paper were born out of a desire to get promoted. They were invented because ML models helped figure out which set of builds we need to run more accurately at scale.
- xiphias2 8y agoI still hope you got promoted though, you deserve it :)
- pastor_elm 8y agoSounds so much simpler outside the context of a 'research' paper: >When an engineer attempts to land their commit, it gets enqueued on the Submit Queue. This system takes one commit at a time, rebases it against master, builds the code and runs the unit tests. If nothing breaks, it then gets merged into master. With Submit Queue in place, our master success rate jumped to 99%. https://eng.uber.com/ios-monorepo/ https://eng.uber.com/ios-monorepo/
- huac 8y agosubmit queue makes sense and is used by lots of people, it's the "machine learning" which is applied to choosing commits to enqueue which I found to be interesting. if the master success rate was already 99% in 2017, with just submit queue, why build the complex ML stuff?
- lallysingh 8y agoThe Submit Queue is the name of the complex ML stuff.
- novok 8y agoAs far as I understand, what your describing is a simple build only one commit at a time submit queue. What they are describing here is to detect if items do not conflict beyond a simple merge conflict and build & commit them simultaneously, increasing the throughput of the submit queue system.
- sundargates 8y agoThe way we achieved the master success rate of 100% at scale was by using the techniques that we describe in the paper. The blog doesn't go into details on Submit Queue and how it works. Just to clarify, the ML models are used to predict the prob. that a given change will succeed against master as well as the prob. of conflict between changes.
- numlocked 8y agoIf there are 1000 commits per day (which wouldn't be that many), that's 10 master breaks per day.