This is unfortunately always going to be a problem with a developing space. Pachyderm was designed with this issue in mind though. While I don't think we've tot

This is unfortunately always going to be a problem with a developing space. Pachyderm was designed with this issue in mind though. While I don't think we've totally eliminated, or can totally eliminate it, we have done some things to mitigate it. Pachyderm pipelines run your code in a docker container that you define which reads data from the local filesystem. It's designed to be simple and similar to the interface you'd use when playing around with data on your laptop. User's generally have a very easy time migrating existing code into Pachyderm, since often code is already reading from the local filesystem and just needs to be pointed at a new path. Migrating to a different system is normally a little more complicated because it requires integrating whatever that new systems data interface is. Although in another system where you just read from local disk the migration should be quite simple.