Not OP but I was just looking through their repo and I've done something similar in the past for a different application. We both used Blender for generating th

Not OP but I was just looking through their repo and I've done something similar in the past for a different application. We both used Blender for generating the rendered synthetic data. Blender has a really simple python API that was originally made for simple scripts but you can run it in a headless mode. You start with a base scene that has all of your objects and use the api to randomly generate samples and render them out to a file.