4 ms·
(author here) I agree it would be fantastic to have sub second deploys! Doing this for user specified Python environments is challenging in different ways than
by shalabhc 4y ago
(author here)
I agree it would be fantastic to have sub second deploys! Doing this for user specified Python environments is challenging in different ways than doing it for a JS SDK like Workers. Note that just provisioning a GitHub runner takes about 10s, before any deploy code even starts up. In theory we could rsync the code directly to the code server and reload it. But any of these options require bigger architectural changes.
Also you can already have a local environment setup for faster iteration and use this for the shared environments with other developers, where the speedup is still great to have.
- MuffinFlavored 4y ago> Doing this for user specified Python environments Who is using this/asking for this/why?
- shalabhc 4y agodagster.io is an open source Python library for building data pipelines. When using Dagster your project may depend on other Python data analysis and warehouse libraries like pandas, snowflake, pyspark and so on. Each org's requirements for their data orchestration will be different. When executing the pipelines in Dagster Cloud, we need to replicate their Python environment.