I recommend looking at buildbot[1]. It can be tailored towards your needs. I developed and maintain one for scummvm [2]. It has a steep learning curve, as with

I recommend looking at buildbot[1]. It can be tailored towards your needs. I developed and maintain one for scummvm [2]. It has a steep learning curve, as with any sufficiently advanced system. [1] https://buildbot.net/ https://buildbot.net/ [2] https://john.scummvm.org https://john.scummvm.org