3 ms·
I take advantage of I/O being parallel in Python in my mazzle continuous integration pipeline tool. I'm not sharing mutable state. I spin up a graph of python
by samsquire 5y ago
I take advantage of I/O being parallel in Python in my mazzle continuous integration pipeline tool. I'm not sharing mutable state.
I spin up a graph of python Threads and each joins others in a graph. This way we can run graphs in parallel. See this graph - the parts that look like this:
dependency -> {parallel1; parallel2; parallel3} -> postparallel
parallel1, parallel2, parallel3 can run in parallel in a separate python thread because the IO is parallel.
postparallel joins parallel1, parallel2, parallel3 and waits for them all to complete.
Where parallel1-3 is things like ansible, packer (slow), AMI builds, chef runs etc.
https://github.com/samsquire/mazzle-starter/blob/master/architecture.png https://github.com/samsquire/mazzle-starter/blob/master/arch...