3 ms·
The nearest I've come to working in an environment like this is a system called Quartz developed at Bank of America. It's a massive set of infrastructure and co
by simonh 6y ago
The nearest I've come to working in an environment like this is a system called Quartz developed at Bank of America. It's a massive set of infrastructure and code based on Python. It consists of a variety of services such as a distributed synchronised hierarchical object store, a set of compute grids, a web server farm, etc. All the code is in a massive code repository and is world readable by any developer in any team, with the configuration for everything in a single config hierarchy based on YAML.
To create a batch process you just commit the code, commit the config for where and when you want it to run and on which host group, and you're done. You can even develop desktop apps running a copy of the runtime locally. Because everything is Python (obviously low level libraries were done in C or C++) and exposed to python APIs, and all the Python code was available it was very easy to develop interoperable applications.
Of course there was huge duplication, but it was an incredibly fun system to work on. I really miss it. The previous iteration of it was actually called Athena and developed at JP Morgan by the same team, who were later hired away by BoA. Even before that they developed the original version at Goldman Sachs, but that wasn't based on Python.