10 ms·
Really interesting read. From what's described: Walpole (distributed job runner), Dagger (DAG that recalculates when dependencies change), Barbara (global key v
by forgotmyoldacc 5y ago
Really interesting read. From what's described: Walpole (distributed job runner), Dagger (DAG that recalculates when dependencies change), Barbara (global key value store) and monorepo/fast deployment, its not so different from some big tech companies.
- databag 5y agoAt JPM, Athena: Walpole = Bob job runner Dagger = pixies Barbara = hydra The python 3 migration is still ongoing
- markus_zhang 5y agoJesus I can imagine how difficult it is to migrate the whole solution to Python3.
- tstordyallison 5y agoIndeed. Thankfully, I think it can be said the light is now strongly visible at the end of the tunnel on Python 3 (but it’s not completely done).
- timkpaine 5y agoYou can see how it's tied together in this presentation https://youtu.be/M9o9SF5-Pzw https://youtu.be/M9o9SF5-Pzw