5 ms·

Author of Mutagen here, happy to answer any questions you might have, whether it's about its use with Docker, SSH, its historical integration into Docker Deskto

Author of Mutagen here, happy to answer any questions you might have, whether it's about its use with Docker, SSH, its historical integration into Docker Desktop, or its related Mutagen Compose[0][1] project. [0]: https://github.com/mutagen-io/mutagen-compose https://github.com/mutagen-io/mutagen-compose [1]: https://mutagen.io/documentation/orchestration/compose https://mutagen.io/documentation/orchestration/compose

slightwinder 5y ago
How robust is the handling? Can it tolerate unreliable network? Lost connections and hiccups?
- xenoscopic 5y ago
  It can tolerate disconnection very reliably, and it will automatically reconnect to synchronization and forwarding endpoints as soon as possible. Its synchronization algorithm in particular (which is essentially a (repeated) three-way merge with rsync-style differential file transfers) is designed for safety and robust tracking of the changes that it has propagated. It's also capable of resuming file transfers once it reconnects.
lifty 5y ago
Thank you for this great project! I’ve been using it for a while now and it has been rock solid and easy to use. Cheers!
vorpalhex 5y ago
Thank you for building this - it appears to solve a real problem that has been a pain in my butt for a couple of years. What are the plans for funding the project? It looks like everything is OSS now, will there be any closed source elements in the future?
- xenoscopic 5y ago
  To be totally transparent: funding is a bit of an open question, although there's no risk of Mutagen disappearing in the short term. Mutagen was actually part of YC's S19 batch, and it still has a significant amount of runway from that, but also some revenue from contracting work. I really want to keep as much of Mutagen as FOSS as possible, ideally MIT licensed. I did just add a small portion of code under the SSPL, which I might experiment with dual licensing to SaaS embedders of Mutagen (since it's really only useful in cases where Mutagen is being embedded in other tooling), but even that I wanted to keep open source for other FOSS projects embedding Mutagen. In the near term, I have some ideas about plugins and tooling that I want to build on top of Mutagen that will probably be closed source, but those will be separate entities from Mutagen itself and the aim will be to avoid compromising on any functionality that belongs in the core of Mutagen.
pchico83 5y ago
Hi Jacob. I am one of the founders of Okteto (https://okteto.com/ https://okteto.com/), a remote development platform for Compose and Kubernetes applications. We use Syncthing to sync code between the developer laptop and pods running in Kubernetes. I would love to know your thoughts on the strengths and weak points of Mutagen vs Syncthing for this use case. Thanks!
- pchico83 5y ago
  Also, how hard would it be to run mutagen on web assembly?
  - xenoscopic 5y ago
    It should be possible, especially since Mutagen is already built for almost all of Go's supported architectures. The biggest issue would just be implementing the race-free filesystem traversal that's used on Windows and POSIX (or potentially living without it given that WASM would probably provide sufficient sandboxing). In any case, most of the work would take place in Mutagen's `filesystem` package, where those syscall equivalents would have to be figured out. The rest of Mutagen should compile without modification. You'd also need to define a transport for Mutagen to use, which would depend on the exact mode of operation you're looking at, but that's the easier problem to solve. Mutagen v0.15 is going to be focused on extensibility (including custom transports), so something like this may become a reality soon.
- xenoscopic 5y ago
  Sure, that's a great question. I'll preface my response by saying that I'm a huge fan of Syncthing (and that Mutagen's use cases form a Venn diagram with those of Syncthing (as well as tools like rsync)). Technologically, all three of these tools are very similar in terms of using the rsync differential transfer algorithm, but their architectures and primary use cases differ. I think the core differentiators with Mutagen are: Development-oriented: Mutagen's sync configuration is primarily focused on development, so it adds more granular controls for things like uni-/bi-directionality, conflict resolution, ignores, symbolic link handling, etc. It also has a permission propagation model that's focused on things like cross-platform executability propagation and preservation (i.e. between Windows and POSIX), as well as operating in multi-service environments where many different process UIDs/GIDs might be in play. It also handles weird filesystem quirks (like macOS Unicode decomposition). Low-latency: Mutagen's goal is to reduce the latency of sync cycles (i.e. time from local edit to change reflection on the remote) to an imperceptible level. It uses a lot of tricks to try to do this, but the goal is really to use the absolute best filesystem watching mechanism for each platform and to integrate that tightly with the sync loop. On Linux, for example, Mutagen is now starting to experiment with the recently revamped fanotify[0] API to get highly scalable but low-latency watching (as opposed to the emulated and janky recursive watching emulation with inotify that most tools use). It also uses tricks like rsync-diffing of the metadata snapshots that it transfers to get latency as low as possible. The eventual goal is to reach sub-100ms sync cycles for multi-GB codebases, and I think that's pretty close. Git-like sync: Conceptually, Mutagen's sync algorithm is like a filesystem watcher + repetitive three-way Git merge (with the difference being that file transfers are deltified and the merging (potentially) affects both endpoints). This means it tracks content in a manner very similar to Git's CAS and branches, which is a little different than the way that Syncthing does it. This affords (in my opinion) more precise identification of conflicts. Distrust: Mutagen takes a more aggressive approach to mutual distrust between endpoints, working hard to ensure that a malicious endpoint can't read outside the synchronization root on the other endpoint via symbolic links or maliciously crafted paths. It does this by using POSIX *at functions to traverse the filesystem and perform operations. This avoids issues like CVE-2017-1000420. You can harden this further by using unidirectional sync and other configuration options. This makes it well-suited to cases where multiple users might be syncing to file storage on a shared system (say on a SaaS platform) (though, at least in that case, you can protect yourself and users with the filesystem namespacing afforded by containers). One-sided installation and flexible topology: Mutagen's primary M.O. is injecting small "agent" binaries to remote systems via a copy mechanism (such as `scp` or `docker cp`), so you don't have to manually install it on both endpoints. This is less important to "full stack" cases like Okteto, where your tooling can handle the setup of Syncthing on the remote, but it makes working directly over SSH or in ephemeral containers significantly more convenient. And Mutagen's architecture is also really flexible, allowing it to sync files and forward traffic between any combination of local and remote endpoints (including remote-to-remote, proxied via the local Mutagen daemon). Command-based transports: Mutagen uses the standard I/O streams of commands like `ssh` and `docker exec` as its transport (similar to tools like Git or rsync), making it easier to target remote environments with your existing tooling and configuration. Again, this is less of an issue for a case like Okteto's, but is useful in the standalone case. Network forwarding: This is outside the scope of sync, but Mutagen offers OpenSSH-style TCP/UDS forwarding (with the difference from OpenSSH being that Mutagen's forwarding is persistent and managed by a background daemon). This offers support for doing things like forwarding a local socket to a remote Docker daemon over SSH, and then forwarding web application traffic over that underlying forwarding by using Mutagen over `docker exec` (or reverse forwarding, or forwarding between two remotes and bridging them via your laptop, or loads of other crazy shenanigans). I hope that clarifies things a bit. Ping me via email if you want an expanded comparison. [0]: https://man7.org/linux/man-pages/man7/fanotify.7.html https://man7.org/linux/man-pages/man7/fanotify.7.html

slightwinder 5y ago

How robust is the handling? Can it tolerate unreliable network? Lost connections and hiccups?

xenoscopic 5y ago
It can tolerate disconnection very reliably, and it will automatically reconnect to synchronization and forwarding endpoints as soon as possible. Its synchronization algorithm in particular (which is essentially a (repeated) three-way merge with rsync-style differential file transfers) is designed for safety and robust tracking of the changes that it has propagated. It's also capable of resuming file transfers once it reconnects.

lifty 5y ago

Thank you for this great project! I’ve been using it for a while now and it has been rock solid and easy to use. Cheers!

vorpalhex 5y ago

Thank you for building this - it appears to solve a real problem that has been a pain in my butt for a couple of years. What are the plans for funding the project? It looks like everything is OSS now, will there be any closed source elements in the future?

xenoscopic 5y ago
To be totally transparent: funding is a bit of an open question, although there's no risk of Mutagen disappearing in the short term. Mutagen was actually part of YC's S19 batch, and it still has a significant amount of runway from that, but also some revenue from contracting work. I really want to keep as much of Mutagen as FOSS as possible, ideally MIT licensed. I did just add a small portion of code under the SSPL, which I might experiment with dual licensing to SaaS embedders of Mutagen (since it's really only useful in cases where Mutagen is being embedded in other tooling), but even that I wanted to keep open source for other FOSS projects embedding Mutagen. In the near term, I have some ideas about plugins and tooling that I want to build on top of Mutagen that will probably be closed source, but those will be separate entities from Mutagen itself and the aim will be to avoid compromising on any functionality that belongs in the core of Mutagen.

pchico83 5y ago

Hi Jacob. I am one of the founders of Okteto (https://okteto.com/ https://okteto.com/), a remote development platform for Compose and Kubernetes applications. We use Syncthing to sync code between the developer laptop and pods running in Kubernetes. I would love to know your thoughts on the strengths and weak points of Mutagen vs Syncthing for this use case. Thanks!

pchico83 5y ago
Also, how hard would it be to run mutagen on web assembly?
- xenoscopic 5y ago
  It should be possible, especially since Mutagen is already built for almost all of Go's supported architectures. The biggest issue would just be implementing the race-free filesystem traversal that's used on Windows and POSIX (or potentially living without it given that WASM would probably provide sufficient sandboxing). In any case, most of the work would take place in Mutagen's `filesystem` package, where those syscall equivalents would have to be figured out. The rest of Mutagen should compile without modification. You'd also need to define a transport for Mutagen to use, which would depend on the exact mode of operation you're looking at, but that's the easier problem to solve. Mutagen v0.15 is going to be focused on extensibility (including custom transports), so something like this may become a reality soon.
xenoscopic 5y ago
Sure, that's a great question. I'll preface my response by saying that I'm a huge fan of Syncthing (and that Mutagen's use cases form a Venn diagram with those of Syncthing (as well as tools like rsync)). Technologically, all three of these tools are very similar in terms of using the rsync differential transfer algorithm, but their architectures and primary use cases differ. I think the core differentiators with Mutagen are: Development-oriented: Mutagen's sync configuration is primarily focused on development, so it adds more granular controls for things like uni-/bi-directionality, conflict resolution, ignores, symbolic link handling, etc. It also has a permission propagation model that's focused on things like cross-platform executability propagation and preservation (i.e. between Windows and POSIX), as well as operating in multi-service environments where many different process UIDs/GIDs might be in play. It also handles weird filesystem quirks (like macOS Unicode decomposition). Low-latency: Mutagen's goal is to reduce the latency of sync cycles (i.e. time from local edit to change reflection on the remote) to an imperceptible level. It uses a lot of tricks to try to do this, but the goal is really to use the absolute best filesystem watching mechanism for each platform and to integrate that tightly with the sync loop. On Linux, for example, Mutagen is now starting to experiment with the recently revamped fanotify[0] API to get highly scalable but low-latency watching (as opposed to the emulated and janky recursive watching emulation with inotify that most tools use). It also uses tricks like rsync-diffing of the metadata snapshots that it transfers to get latency as low as possible. The eventual goal is to reach sub-100ms sync cycles for multi-GB codebases, and I think that's pretty close. Git-like sync: Conceptually, Mutagen's sync algorithm is like a filesystem watcher + repetitive three-way Git merge (with the difference being that file transfers are deltified and the merging (potentially) affects both endpoints). This means it tracks content in a manner very similar to Git's CAS and branches, which is a little different than the way that Syncthing does it. This affords (in my opinion) more precise identification of conflicts. Distrust: Mutagen takes a more aggressive approach to mutual distrust between endpoints, working hard to ensure that a malicious endpoint can't read outside the synchronization root on the other endpoint via symbolic links or maliciously crafted paths. It does this by using POSIX *at functions to traverse the filesystem and perform operations. This avoids issues like CVE-2017-1000420. You can harden this further by using unidirectional sync and other configuration options. This makes it well-suited to cases where multiple users might be syncing to file storage on a shared system (say on a SaaS platform) (though, at least in that case, you can protect yourself and users with the filesystem namespacing afforded by containers). One-sided installation and flexible topology: Mutagen's primary M.O. is injecting small "agent" binaries to remote systems via a copy mechanism (such as `scp` or `docker cp`), so you don't have to manually install it on both endpoints. This is less important to "full stack" cases like Okteto, where your tooling can handle the setup of Syncthing on the remote, but it makes working directly over SSH or in ephemeral containers significantly more convenient. And Mutagen's architecture is also really flexible, allowing it to sync files and forward traffic between any combination of local and remote endpoints (including remote-to-remote, proxied via the local Mutagen daemon). Command-based transports: Mutagen uses the standard I/O streams of commands like `ssh` and `docker exec` as its transport (similar to tools like Git or rsync), making it easier to target remote environments with your existing tooling and configuration. Again, this is less of an issue for a case like Okteto's, but is useful in the standalone case. Network forwarding: This is outside the scope of sync, but Mutagen offers OpenSSH-style TCP/UDS forwarding (with the difference from OpenSSH being that Mutagen's forwarding is persistent and managed by a background daemon). This offers support for doing things like forwarding a local socket to a remote Docker daemon over SSH, and then forwarding web application traffic over that underlying forwarding by using Mutagen over `docker exec` (or reverse forwarding, or forwarding between two remotes and bridging them via your laptop, or loads of other crazy shenanigans). I hope that clarifies things a bit. Ping me via email if you want an expanded comparison. [0]: https://man7.org/linux/man-pages/man7/fanotify.7.html https://man7.org/linux/man-pages/man7/fanotify.7.html