2 ms·
Thank you! That's a great question. First and foremost, dstack treats artifacts are 1st class citizens. Here's the basic way of using them: A workflow may pro
by cheptsov 4y ago
Thank you! That's a great question.
First and foremost, dstack treats artifacts are 1st class citizens.
Here's the basic way of using them:
A workflow may produce output files to a local folder. In your workflow declaration, you can mark what folders to treat as output artifacts.
Then, dstack would save output artifacts automatically, and you'll be able to reuse them via the unique name of the run, or a user tag assigned to this run.
All artifacts are stored in the S3 bucket that is configured for dstack.
dstack is capable of syncing artifacts at start//end of the workflow or mount artifact folders via FUSE (of course if that is needed).
Each artifact is stored using the following path: <s3 bucket>/artifacts/<run name>/<job name>
A run can have multiple jobs, e.g. if it's a distributed workflow.
In future, we also think of providing a high-level Python API for accessing/storing artifacts.
Please share your thoughts and feedback!