11 ms·

Ask HN: Is anyone successfully self-hosting Firefox Sync?

366 points by hosteur 4y ago

I use Firefox as my primary browser, and I would absolutely love to be able to use the Firefox Sync functionality to sync bookmarks, etc. across several devices. However, I do not want to risk storing absolutely sensitive data (browsing history, auth cookies, credentials, etc) at some third party. Everytime this comes up in HN comments, someone says that Firefox Sync can be self-hosted. And while this was true in the older versions of Firefox, it seems to me that self-hosting is currently neither supported nor documented. And it has been like this for at least since 2020[1][2]. It seems self-hosting is not a priority for Mozilla. Is anyone actually self-hosting Firefox Sync? If so, how do you do it? How is it working for you? Any issues to be aware of? [1]: https://github.com/mozilla-services/syncstorage-rs/issues/681 (opened on Jun 22, 2020) [2]: https://github.com/mozilla-services/syncstorage-rs/issues/550 (opened on May 26, 2020)

tommica 4y ago
Maybe this article can help you decide? https://homegrowntechie.com/self-host-your-browser-data/ https://homegrowntechie.com/self-host-your-browser-data/
- pepa65 4y ago
  This one seemed to be doing more: https://github.com/crazy-max/docker-firefox-syncserver https://github.com/crazy-max/docker-firefox-syncserver
- pepa65 4y ago
  Just tried that, it's just broken (at least, those instructions).
arkadiyt 4y ago
> However, I do not want to risk storing absolutely sensitive data (browsing history, auth cookies, credentials, etc) at some third party. FWIW Firefox says they do not have the capability to read your data: https://hacks.mozilla.org/2018/11/firefox-sync-privacy/ https://hacks.mozilla.org/2018/11/firefox-sync-privacy/
- deleted 4y ago
  [deleted]
- gorgoiler 4y ago
  Unfortunately, this isn’t really true. The party which holds your data is also the party which ships evergreen versions of the software used to decrypt that data. There’s nothing to stop the government in Firefox’s jurisdiction from bullying them into adding a backdoor. We can assume the bullying is ongoing. I like to imagine it as having a safe deposit box provided by a local bank. I have the only key and have proven that to be the case. I also store the key at home in another safe which is regularly serviced by bank employees. While they probably don’t abuse their position in the latter to access the former, it would be completely wrong to say they do not have the capability to get at my stuff. All that aside, I love FF sync and rely on it everyday for work stuff. It is like magic and I’m very grateful for the feature. I am realistic about the NSA’s level of interest in my work calendar and the threat they pose to my day to day life.
  - arkadiyt 4y ago
    > The party which holds your data is also the party which ships evergreen versions of the software used to decrypt that data. There’s nothing to stop the government in Firefox’s jurisdiction from bullying them into adding a backdoor. Hosting your own sync server doesn't prevent this.
    - tinus_hn 4y ago
      The government can’t bully you into decrypting your own data or releasing your encrypted data without you knowing.
      - robjan 4y ago
        If Firefox is backdoored, as per GP's hypothetical situation, your data can be sent to a sync server other than the one you configured. The only realistic way to stop it would be to only use a specific audited version of FF. If that's the case then there's no need to use your own sync server because you already know it's not backdoored.
balu_ 4y ago
unfortunately not anymore, in the early days of firefox sync back when it was called wave it was easy to selfhost. Just throw some[1] php files to a host and it worked. Then mozilla changed it to use multiple services (auth, ...) it got difficult and the documentation wasn't easy to access. At that time i stoped selfhosting & using it :( [1]: https://github.com/balu-/FSyncMS https://github.com/balu-/FSyncMS
williamvds 4y ago
I've self-hosted the original Firefox syncserver[0] for a couple of years now. It's a now unmaintained Python 2 + gunicorn web app. It still uses the Firefox identities service, since I haven't bothered to try setting that one up myself. That means I need a Firefox/Mozilla account even with a self-hosted sync server, which isn't ideal. Note that Mozilla do provide a script to delete all your data from their servers[1], in case you were using Firefox sync before going self-hosted. I originally went through the setup instructions manually. Now I've created a Nix package[2] and NixOS module for it[3], which was an adventure by itself. I can now easily reproduce the setup, which I did while migrating my webserver to a different provider. As for why I didn't try syncstorage-rs, it's rather petty, but the old syncserver was working well for me, and it doesn't yet support SQLite. [0] https://github.com/mozilla-services/syncserver https://github.com/mozilla-services/syncserver [1] https://github.com/mozilla-services/syncserver#removing-mozilla-hosted-data https://github.com/mozilla-services/syncserver#removing-mozi... [2] https://git.sr.ht/~williamvds/config/tree/master/item/nix/pkgs/firefox-syncserver https://git.sr.ht/~williamvds/config/tree/master/item/nix/pk... [3] https://git.sr.ht/~williamvds/config/tree/master/item/nix/modules/firefox-syncserver.nix https://git.sr.ht/~williamvds/config/tree/master/item/nix/mo...
- wkat4242 4y ago
  The first few lines in the repo you mention illustrate the problem: > Note that this repository is no longer being maintained. Use this at your own risk, and with the understanding that it is not being maintained, work is being done on its replacement, and that no support or assistance will be offered. And its replacement syncserver-rs is in this forever-unfinished state so they've effectively killed self-hosting. At least from a support standpoint.
  - williamvds 4y ago
    Good point, I'll update the comment to indicate syncserver is now unmaintained. Of course, everything self-hosted is at my own risk etc. And official support is a luxury in this kind of FOSS, I generally don't count on it. But since it works for me, it may be worth sharing the details.
captn3m0 4y ago
I faced enough issues that I switched to the Mozilla endpoints. Problems ranged from no clear documentation for configuration, unclear resource usage, no documented method to host identity as well. My one hope was to get my data exported via SQLite for fun stuff, as an automated export. But the data format was also not well documented, so that never happened.
cricalix 4y ago
Anecdotally, I miss the days of Netscape combined with Netscape Enterprise Server. Ran it at college, and you could log in to any Netscape browser and all your bookmarks appeared. Then NES died, Netscape became Mozilla and it all went away. I see there are bookmark sync extensions, but only bookmarks (and I didn't check if those extensions allow self hosting). I suppose you could always run Firefox portable from a location that was Dropbox/Nextcloud/syncthing/whatever synced. Just don't run two instances at once, or the SQLite DBs will have a bad time?
- hosteur 4y ago
  > I suppose you could always run Firefox portable from a location that was Dropbox/Nextcloud/syncthing/whatever synced. Just don't run two instances at once, or the SQLite DBs will have a bad time? I do something to that effect. However, it does not come without problems. Running parallel instances is one of them... Also, cross platform is a problem.
  - cricalix 4y ago
    Hypothetically, since I have no way to test this easily right now, you only need to run the profile directory in a synced location and run the exe from a platform-specific portable directory.
- mvdwoord 4y ago
  I miss Xmarks... I believe it was killed off completely when lastpass was aquired by Logmein, a couple of years ago. Xmarks really hit the spot for me for syncing bookmarks. It did only that, across browsers, across machines, and because It was limited to bookmarks sync, I had no qualms about using the same account on employer provided machines. https://en.wikipedia.org/wiki/Xmarks_Sync https://en.wikipedia.org/wiki/Xmarks_Sync
skozharinov 4y ago
I self-host syncserver in Docker, it is relatively easy to set up. The main issue is that it would not tell you that it doesn't work, instead it would silently leave a log file in your browser (about:sync-log). Once it is up and running, however, it works flawlessly. You also have to consider that syncserver is unmaintained and uses Python 2, which has long been EOL.
Mikescher 4y ago
Shameless plug: A while ago I wrote a cli for firefox sync [1] (in my case mostly to automate some stuff when I create bookmarks). It does not really solve your problem for browser-history/credentials etc. But as long as you use a client that you verified you can be kinda sure the data is E2E encrypted and can't be read by Mozilla. Also on a side note: To this day the weave protocol is one of the more (needlessly) complex protocols I ever implemented and I'm still not sure what problems some of the design decisions solve... [1] github.com/mikescher/firefox-sync-client
- Wicher 4y ago
  This looks very useful, thanks for creating!
- darylfritz 4y ago
  Does this have the ability to read the open tabs that are synced from all devices?
- deleted 4y ago
  [deleted]
gpchelkin 4y ago
https://kmj.at/en/Setup_Firefox_Sync_standalone_or_Firefox_FXA_Account_Server_and_Sync_Server_on_Debian_Bullseye_using_Docker/ https://kmj.at/en/Setup_Firefox_Sync_standalone_or_Firefox_F...
- cricalix 4y ago
  I think it's worth noting that that document links to the Github for the sync server, and that page says "Note that this repository is no longer being maintained. Use this at your own risk, and with the understanding that it is not being maintained, work is being done on its replacement, and that no support or assistance will be offered." So yes, you can probably get it running in docker, but bugs etc won't be fixed unless you're comfortable working on older Python code.
Tepix 4y ago
What other self-hosted bookmark sync solutions exist that work with Firefox (and perhaps other browsers)?
- FireInsight 4y ago
  I used to use xbrowsersync [0] for cross-browser sync and rn I'm using raindrop [1]. Xbrowsersync is self-hostable, but raindrop is not. Both are FOSS. There's also a big list of different projects here: https://github.com/awesome-selfhosted/awesome-selfhosted#bookmarks-and-link-sharing https://github.com/awesome-selfhosted/awesome-selfhosted#boo... [0] https://www.xbrowsersync.org/ https://www.xbrowsersync.org/ [1] https://raindrop.io/ https://raindrop.io/
- mackrevinack 4y ago
  ive been using floccus for a few months now and its been working ok so far. it syncs to the webdav server on my synology but there's also the option to sync it using decsync (which is sort of like a local webdav server for each device) and some other sync service like syncthing or dropbox to sync the files. floccus has an android app or it can also be installed as a normal extension with the kiwi browser. i tried xbrowsersync first but i had some issue when i used it with vivaldi where a new copy of my bookmarks showed up in the "deleted bookmarks" section every time a sync happened, and i couldn't see any way of fixing that either which is why i went with floccus in the end
  - seabrookmx 4y ago
    +1 for Floccus I run a simple little docker container (found the image on github)[1] to host the webdav server myself, since I don't run Nextcloud or anything else that exposes webdav. It works quite well for me. [1]: https://github.com/marlluslustosa/Floccus-WebDavDocker https://github.com/marlluslustosa/Floccus-WebDavDocker
- johnchristopher 4y ago
  I am using https://github.com/sissbruecker/linkding https://github.com/sissbruecker/linkding at the moment.
wasmitnetzen 4y ago
Yes, I'm running it with the mozilla/syncserver docker image[1], but still using the hosted identity service. The setup procedure is a bit fickle, the error handling on Firefox' side is quite bad, sometimes the first sync times out when you have a larger data set, and you need to manually enable each sync type to reduce the size. But once it's up and running, I don't really have any issues. I do hope that syncstorage-rs will become usable eventually and hopefully improve performance. Note: contrary to your post, cookies are not synced. [1]: https://hub.docker.com/r/mozilla/syncserver https://hub.docker.com/r/mozilla/syncserver
- deleted 4y ago
  [deleted]
- heywoodlh 4y ago
  What is gained from self hosting the container? Are you able to sync more data than Firefox's official Sync service? Less reliance on a third party service? Would love to understand the value-add. Their docs[0] make it seem like little would be gained from a privacy/security perspective as it seems pretty secure/private (at least to me). [0] https://support.mozilla.org/en-US/kb/how-firefox-sync-keeps-your-data-safe-even-if-tls-fails https://support.mozilla.org/en-US/kb/how-firefox-sync-keeps-...
  - eqvinox 4y ago
    You could put your self-hosted container in a self-hosted VPN to add a layer of protection. But honestly, while encryption theoretically can make any data "safe" to hand to a 3rd party, it's just good "data hygiene" to keep it under your own control. If someone steals the Firefox sync database now, and some part of the encryption scheme is later broken by e.g. quantum computers, the thieves could retroactively decrypt your sync account data. And there's also the question of metadata - when does your Firefox send data to sync, and how much? Are you sure there's nothing to glean from that?
    - smarnach 4y ago
      You are making the assumption that you can secure your own instance of Sync better than Mozilla can secure theirs.
the_third_wave 4y ago
No, I used to host the previous version but when that stopped working (including the workarounds which kept it going for a bit longer) I ceased using it. While the sync server itself is not that much of an effort to get going, the auth server is. I do not use external auth services when I can avoid it so I chose to forego on using sync instead of creating a "Firefox account". If I ever find an easier way to get the auth server running and if I still use Firefox by that time I might start using it again but until that time it is a no-go. To Mozilla: fire your overpaid activist CEO Mitchell Baker and use the resulting funds to hire (back) the developers she got rid of, clean up the mess, get rid of the social justice nonsense and take back your place as a browser development organisation instead of a 'colours of change' peddler.
3np 4y ago
Yes, and I went the whole mile. I wrote a couple of comments on it previously[0][1], which at least lay out the required images for a full stack. As you can see it doesn't use the newer release but it still runs fine on latest FF. Though you probably want the new version. Can't imagine it being significantly worse to get up and running now. TLDR: If you're OK with piggybacking on them for auth, it's straightforward, easy and lightweight. If you want to be fully self-sufficient and leak no data you need to either implement a replacement for fxa (seriously this would be appreciated and maybe less effort if you're coming in fresh) or host that whole machinery of microservices - this is where the head-scratching starts and you need some dedication for it. I encourage other people to do it, and do go to whichever is more appropriate of the Matrix room or the GitHub Issues and engage when you hit things that are out of date, unclear, or broken. [0]: https://news.ycombinator.com/item?id=30315816 https://news.ycombinator.com/item?id=30315816 [1]: https://news.ycombinator.com/item?id=30727935 https://news.ycombinator.com/item?id=30727935
mbwgh 4y ago
Why does every "Ask HN" I see come with a grey text color? Is this a stylistic thing or do people take issue with people asking questions and downvote?
- aamargulies 4y ago
  HN does this purposefully to show its preference for external content rather than self posts.
  - KronisLV 4y ago
    I wonder if there could be an alternative way of getting the point across, as opposed to having low contrast text, which some tools report as an accessibility problem: https://wave.webaim.org/report#/https://news.ycombinator.com/item?id=34674569 https://wave.webaim.org/report#/https://news.ycombinator.com... But other than that, it's an interesting design choice.
  - pancrufty 4y ago
    HN does this because they don’t care. It wouldn’t take much to make this site usable, but they refuse to add 5 lines of CSS to make the site work in dark mode.
    - SturgeonsLaw 4y ago
      https://userstyles.org/styles/browse/ycombinator https://userstyles.org/styles/browse/ycombinator
      - pancrufty 4y ago
        Doesn't sound like a good default. I used those before, but I shouldn't have to to make a website barely usable.
  - groestl 4y ago
    Aaaaah facepalm. Using this for a decade, never had a clue.
yownie 4y ago
I found the lockwise app very useful for FF sync and have no idea why they shitcanned it. I don't really understand how Mozilla continues to function at all as an org.
- mnoorenberghe 4y ago
  The functionality is (mostly) built into Firefox now so there’s little need for a separate app. You can enable native app autofill for both Android and iOS. There are a few things that should still be done to improve the experience but it doesn’t seem like a priority now.
berndinox 4y ago
Vaultwarden for Passwordsync and Wallabag for Bookmarks. Con: no „real“ Firefox Sync - Pro: no „real“ Firefox Sync - it works on any device with any browser, data is self-hosted
- NoboruWataya 4y ago
  I set up Wallabag recently for this very purpose. My initial impression is that it mainly focuses on fetching and storing the content itself, rather than giving you an easy to navigate bookmark library. So it's great for, eg, saving long-form articles and blog posts, but not so good for bookmarking interactive/dynamic content.
  - berndinox 4y ago
    True, its better suited for saving „Text“. I dont have that much bookmarks tbh
nottorp 4y ago
I've got a related question but about more restricted capabilities: I'd like to sync just bookmarks without joining the "Firefox ecosystem" or any other ecosystem. Is there some bookmark sync extension with a self hosted sync solution or that will just sync via a (service similar to) Dropbox folder? Edit: syncing via a git repo would be fine for example :)
- xnickb 4y ago
  xbrowsersync
- seabrookmx 4y ago
  floccus is an option.
NilsIRL 4y ago
Firefox Sync uses send-to-end encryption. So self hosting isn't as necessary.
- fIREpOK 4y ago
  Until they find a bug in the self hosting software
asddubs 4y ago
i just copy the .mozilla folder over whenever i install a new device
- doubled112 4y ago
  Do you not use bookmarks? Or are yours so stable it isn't any work to keep up? This makes tons of sense to me with extensions and the other profile data, but not that.
  - asddubs 4y ago
    I just let them diverge. Most bookmarks I make are needed for a span of less than a week or I never use them again in the end. It's admittedly kind of a chaotic solution, but it works for how I use the browser.
- dblohm7 4y ago
  That only works when you're copying to the same OS. You run the change of entering into a world of hurt if your new device runs on a different platform.
omani 4y ago
I was thinking about doing this with my own custom setup. Im thinking of putting the folder .mozilla into a gocryptfs encrypted folder and use syncthing to sync it between devices.
charles_f 4y ago
I self host most of my stuff for the reasons you mentioned, and that included Sync for a few years. I never got the auth part to work so I stopped trying and just used the sync server itself. I stopped doing that after some time, it seemed too futile. Errors are hard to debug and you're on your own, plus I was not using it for passwords or CC, so it just wasn't worth it.
- rayusher 4y ago
  What other stuff do you host?
  - deleted 4y ago
    [deleted]
vasvir 4y ago
I had blog about it in case you want to set i up with apache as a proxy. Shameless plug: https://vasvir.wordpress.com/2021/03/30/firefox-sync-server-and-apache-as-a-proxy/ https://vasvir.wordpress.com/2021/03/30/firefox-sync-server-...
egberts1 4y ago
Self-hosting Firefox Sync does not work with Apple iOS-based Firefox. Simply because the URL of Firefox Sync hosting is hardcoded into the Firefox iOS app. Hence thwarted my zeal to self-host Firefox Sync. Heck, there is NOTHING configurable about Firefox iOS, not even an URL config:about.
- scns 4y ago
  I read somewhere that Apple allowed other browsers on the AppStore, no more every browser is Safari with a different skin. Anyone else here who has better information?
  - deleted 4y ago
    [deleted]
- SirGiggles 4y ago
  Have you tried tapping the version number (under the in-app Settings) five times? That reveals an advanced sync settings near the top iirc.
  - egberts1 4y ago
    Not there. But that press five times on the version menu item does not reveal anything useful from a self-hosting POV.
timmit 4y ago
https://send.vis.ee/ https://send.vis.ee/

dodgerdan 4y ago

Then you can’t trust the FF code not to send a copy to their own servers also.

alias_neo 4y ago
Why would you need to trust it? Just block access to their servers on your network. Anyone _that_ concerned can operate a whitelist-only policy on their network, now nothing goes anywhere they don't want it to.
- hosteur 4y ago
  White-listing what addresses a web browser can reach seems to go against the intended use - you know, to browse the web. Edit: Also, I don't think this is as much about trust as it is about avoiding being part of the huge target that the centrally hosted Firefox Sync servers must be. While Firefox developers and Mozilla might be wholly trustworthy, they might become compromised by some configuration error or zero day vulnerability. So someone might steal the sweet sweet treasure that is in the Sync servers. Now, that is encrypted of course, but they might have made a simple error[1] in how encryption is implemented and the encryption may be defeated. [1]: See for example: https://www.schneier.com/blog/archives/2008/05/random_number_b.html https://www.schneier.com/blog/archives/2008/05/random_number...
  - bluGill 4y ago
    If you have something you are that concerned about then you should whitelist everything. Note that the whitelist needs to be one both sides, the server and the users. This along with lots of other security protections, some of which are a lot stronger than a whitelist.
  - parminya 4y ago
    Instead of directly exposing the server to the internet, you could keep it behind a VPN. That way you only need to keep the VPN secure, the VPN serves as a whitelist.
  - alias_neo 4y ago
    > White-listing what addresses a web browser can reach seems to go against the intended use - you know, to browse the web. Agreed, but that was an extreme example. Blocking FFs known addresses should be sufficient with some additional network monitoring just in case. I can't response to the rest of your comment, as my point was specifically about not needing to trust FF/MZ if you're running your own sync server. Once the software is within your network, it is entirely up to you, your tools and your skills to determine what does and does not leave it and to where.

toyg 4y ago

I don't understand. The original was barely 500 lines of code in two files, everything else being packaging machinery. Yes it's python 2, but how hard could have it been to port 500 LOCs to 3.x...? Even assuming some crypto libraries broke, surely there are working equivalents? Instead, they went for a Big Rewrite in Rust, resulting in a much bigger codebase that will be much harder to maintain... and broke it anyway.

acdha 4y ago
Their issue tracker suggests that it’s mostly due to their team being focused on the main Firefox service: https://github.com/mozilla-services/syncserver/issues/264#issuecomment-754784657 https://github.com/mozilla-services/syncserver/issues/264#is... I’m sure it wouldn’t be that hard to port but if they’re committed to a different direction it’s probably best not to give the impression that you should plan to keep using it.
- toyg 4y ago
  Yeah, but even internally it makes little sense. You've replaced a very simple system built on a widely-known language with a massive ecosystem, something you could even call "boring technology" these days (python), with a larger implementation using a niche language with a small ecosystem and a minuscule pool of skilled developers. It's now even more unlikely you'll ever get any patches from outsiders, it will be harder to hire people to maintain it, harder to monitor and expand the system... And this for a critical system. I mean, as a job-security device, I'm sure it works out great. As a strategic business decision, though, it looks debatable as a minimum - and I'm being charitable.
  - acdha 4y ago
    Not sure you could call Rust a niche language any more given how many devices it’s running on, but especially at the organization which created it that seems like an understandable choice. I have a lot of experience with Python but can totally sympathize with people running a highly-visible service prioritizing resource utilization.
    - toyg 4y ago
      > Not sure you could call Rust a niche language any more First comment under "The Adoption of Rust in Business": https://news.ycombinator.com/item?id=34691509 https://news.ycombinator.com/item?id=34691509 A bit further down: https://news.ycombinator.com/item?id=34691893 https://news.ycombinator.com/item?id=34691893
      - acdha 4y ago
        Yes, note that’s .gov where they also have trouble hiring technical staff of every sort. I happen to work in .gov and we have multiple people writing Rust so anecdotes are not probably reliable. This is better - still a self-selected audience but a large one - and it shows around 10%. That’s not huge but I think it’s larger than “niche” given the growing popularity and impact. https://survey.stackoverflow.co/2022 https://survey.stackoverflow.co/2022

pmarreck 4y ago

I think Nix, as struggle-full as it was, was the right choice, here, although I don't see the flake.nix file

williamvds 4y ago
The flake starts at this level: https://git.sr.ht/~williamvds/config/tree/master/item/nix https://git.sr.ht/~williamvds/config/tree/master/item/nix Struggle is a good way to put it! I feel I hit one of Nix's pain points here: packaging software whose dependencies are largely missing from Nixpkgs. On top of that, it was a Python app, and the state of Python packaging is quite a mess. I also feel it Nix was the right choice. My journey was rather harsh, but I do believe in its approach.
- pmarreck 4y ago
  The thing that guts me a little is that Nix is also perfect for software preservation, which unfortunately must include Python 2 apps. But meanwhile they've dropped support for Python 2, which means you either have to go with some clever flake solution or something like what you did. I'm dealing with an app that is breaking every time I try to upgrade a few-years-old version of Node, a version that is no longer on nixpkgs. Right now I just want something that works; I didn't sufficiently budget for ongoing maintenance as the whole site's behind a login.
  - williamvds 4y ago
    Another option is to find a Nixpkgs commit which does contain the version of a package that you want. You can pin and import it as a Flake input, and use the old version of the package provided. So in its own way, Nix and Nixpkgs do serve the purpose of software preservation, as long as you have the full commit history for Nixpkgs. Preservation doesn't imply maintenance however. It's fair for Nixpkgs to delete packages that are no longer supported, as that's a burden on the limited number of maintainers. By recreating the package and module in my own Flake, I've taken on responsibility for maintaining it myself, and not adding any work to the Nixpkgs maintainers. NixOS modules in particular aren't easy to extend, so I ended up copying it entirely. And to be honest, the thought of using an old version of Nixpkgs didn't cross my mind while setting up syncserver. Hopefully it's easier to maintain with poetry2nix now that I've got the package working.