6 ms·

I'd love to see "sync with git" become a standard app feature. Too many app developers realize they need some kind of shared state "in the cloud" and end up wi

I'd love to see "sync with git" become a standard app feature. Too many app developers realize they need some kind of shared state "in the cloud" and end up with a database and a backend server. Many of them would be better off if they had a reliable way to push the state management to a user-controlled space, like a git repo or Dropbox

ChadNauseam 4y ago
That’s a great idea. Although key management might be a problem. Also, you might have conflicts if you try to push from one device after having previously pushed from another device.
- hoherd 4y ago
  Conflicts aren't specific to git. Git offers a history of those conflict resolutions though, which lets you correct them if they were not done right. I used to store my DayOne journal in git, and boy, their cloud sync conflict resolution was sometimes quite destructive. Having the git history showed me that, and allowed me to correct it.
vhanda 4y ago
A few things are lacking. 1. Decent desktop apps to sync the data, not everyone is comfortable hacking a Cron script. I've started with that - https://GitHub.com/GitJournal/git-auto-sync https://GitHub.com/GitJournal/git-auto-sync - but it's in the early stages. 2. For mobile apps, a way to drastically reduce the overhead. Cross compiling libgit2 + deps + NDK is a major pain, and jGit uses more modern java features so you need to target a higher version of Android. (This was in early 2019, when I first started GitJournal) 3. There is no competing with a simple http call vs the complexity of git. Plus the entire onboarding process. 4. Many many apps do write to Dropbox or Google Drive, but for me that's not a win as you aren't in control of your data. 5. While hosting your own git server is arguably easier than self hosting anything else, the majority of users in GitJournal (more than 80% use GitHub, this was about a year ago.) There is also a bias as people who self host are more likely to switch off the analytics. I don't send any message when analytics are switched off, so I don't know. 6. We need end-to-end encryption in Git, like keybase was providing, but open source and hostable in any git repo. I started a proof of concept of a custom remote where I just write all the git objects to another git repo, but haven't gotten very far. Disclaimer: I'm the author of GitJournal.
- spicybright 4y ago
  What's your opinion on SyncThing? I find this checks a ton of these boxes for me, and I find it really easy to get non-techies on board and actually be comfortable using it.
  - nyanpasu64 4y ago
    Syncthing works well when devices can find each other (usually over mDNS) and are online simultaneously. After dealing with an entangled mess of introducers and devices, I decided to setup an old phone running Syncthing-Fork as root (otherwise mDNS doesn't work on Android), use it as a central introducer and always-online file host, and put my shared files there. It works pretty well, but I still run into cases sometimes where one client learns of the existence of files from another device, but the other device is offline and my central server doesn't have copies of them. For locally shared documents, I sync them with Syncthing and edit them on Android in Markor, and it works pretty well.
  - bebna 4y ago
    I gone this route. I got different markdown editors on different devices and didn't want to use a public / cloud hosted git repo for my notes. It works fine, as long you don't get in contact with iOS. I got many devices and often let it sync my phone when I'm at home, because I'm struggling already enough with battery lifetime. I also got it running on one of my dedicated servers, so I got an offsite backup, with systemd you can set it on autostart after an internet connection was established, so I don't really have to manage it anymore and I only see it basically in the logs of the packages that I update.
    - spicybright 4y ago
      That's awesome. I use it pretty much the same, only I sync the root of my phone data too. So I can take a photo on my phone and have it pop up on my computer, ready to use. My only small tip for readers here is be careful treating any node as a safe backup. A device that accidentally deletes everything in a folder will propagate out if you don't notice it happened. But TBH in practice (as a very heavy user for my usecase) it's very rare, and a traditional backup system will cover you perfectly.
enasterosophes 4y ago
The Git protocol is free, and projects like libgit2 make it easy to integrate the protocol into new software. If you feel strongly about it, please consider contacting the purveyors of your favorite software and ask why they don't have Git integration. (Edit: libgit2, not libgit.)
- deleted 4y ago
  [deleted]
gernb 4y ago
sync to git isn't perfect. I use Google Keep, which, like docs, I can have the same document open on multiple machines, start editing on one, finished editing on the other, no "commit/push/fetch/rebase/merge" cycle required. The typical scenario for me is start typing something in and then get interrupted (notification, time based appointment, someone needing my attention), when I get back to entering I might be at a different machine (like started on my phone, got interrupted, then sitting at my computer and remember to finish). I don't see how git sync helps this issue.
- oneeyedpigeon 4y ago
  I don't think the OP was suggesting exposing all those low-level details in the UI. Surely the software would do all of that for you?
remram 4y ago
The Git model is to version everything together. This means that a conflict blocks the entire repo until it's fixed. In many situations, it's better to version separate pages/items/notes separately, and have a conflict to a single note leave the rest of your repo working and syncing.
- YetAnotherNick 4y ago
  Separate git repo for each note will fix this, right? Another possibility you can use some trick like https://stackoverflow.com/a/64810110/936584 https://stackoverflow.com/a/64810110/936584 to prefer one version.
  - vhanda 4y ago
    This is similar to what GitJournal does. It'll never present you with a merge conflict and will instead just make prefer the local changes. It's not perfect, but in the worst case, the wrong option is chosen, but no data is lost since the history is there in Git.
codethief 4y ago
To be honest I'd rather love to see "select folder for storage" become a standard app feature. Then I could just use whatever app I want to synchronize the files (in my case probably Syncthing). Integration of $APP_FEATURES and $SYNC_BACKENDS in apps is a major waste of developer resources and it also means that, as a user, I can never freely choose the app based upon its intrinsic features but also always have to keep synchronization in mind.
- mrtesthah 4y ago
  The iOS editor Runestone takes this approach. I’ve installed Möbius Sync on the device, with both apps pointing at the same folder to create a syncthing conduit with my Mac. It’s fully seamless, private, and free.
dotancohen 4y ago
Git stores the entire history of text files. Very few apps need the history feature. Leaving the feature in place may take much disk space, and actually removing files no longer needed is a pain in git, git is not designed for that use case. I do agree that a standardized sync protocol and file format that could be pointed at any arbitrary server would be terrific. I just disagree that git is that format for many use cases.
- prepend 4y ago
  As a note taker, I’d much rather have the history and deal with the disk space than lose the history and have extra storage. I almost never truly want to delete any note. And most changes are additive so don’t take much extra storage due to versioning. I find it maddening to use a note app that doesn’t have detailed change logs and git’s history and blame is really useful.
  - dotancohen 4y ago
    For notes, of course. I also store my notes in git. But GP mentioned "sync with git" as "a standard app feature", not specific to note apps.
- vendiddy 4y ago
  For text based notes, the storage cost is negligible. I've been using a single git repo to store notes for years. And my entire repo is a few MB.
  - dotancohen 4y ago
    You just gave me an idea to garbage collect, let's see how much space we can save: $ du -sh ~/Notes/ 47M /home/dotancohen/Notes/ $ du -sh ~/Notes/.git 21M /home/dotancohen/Notes/.git $ git -C ~/Notes/.git gc $ du -sh ~/Notes/ 44M /home/dotancohen/Notes/
    - vhanda 4y ago
      It might be better to only measure the size of the `.git` folder, as with it you can reproduce everything else. Essentially, a bare git instance.
pydry 4y ago
I wish we just had save to textual flat file as a more common standard app feature (which is simple/easy) and a plethora of separate apps that did the syncing from that (hard - conflicts/authentication, but only needs to be done once for all apps). I have the git syncing tools but most apps dont save to flat file and the ones that do tend to require hitting export -> backup.
dominotw 4y ago
> Too many app developers realize they need some kind of shared state "in the cloud" and end up with a database and a backend server. This is usually the premium offering( eg: obsidian) in their product.

ChadNauseam 4y ago

That’s a great idea. Although key management might be a problem. Also, you might have conflicts if you try to push from one device after having previously pushed from another device.

hoherd 4y ago
Conflicts aren't specific to git. Git offers a history of those conflict resolutions though, which lets you correct them if they were not done right. I used to store my DayOne journal in git, and boy, their cloud sync conflict resolution was sometimes quite destructive. Having the git history showed me that, and allowed me to correct it.

vhanda 4y ago

A few things are lacking. 1. Decent desktop apps to sync the data, not everyone is comfortable hacking a Cron script. I've started with that - https://GitHub.com/GitJournal/git-auto-sync https://GitHub.com/GitJournal/git-auto-sync - but it's in the early stages. 2. For mobile apps, a way to drastically reduce the overhead. Cross compiling libgit2 + deps + NDK is a major pain, and jGit uses more modern java features so you need to target a higher version of Android. (This was in early 2019, when I first started GitJournal) 3. There is no competing with a simple http call vs the complexity of git. Plus the entire onboarding process. 4. Many many apps do write to Dropbox or Google Drive, but for me that's not a win as you aren't in control of your data. 5. While hosting your own git server is arguably easier than self hosting anything else, the majority of users in GitJournal (more than 80% use GitHub, this was about a year ago.) There is also a bias as people who self host are more likely to switch off the analytics. I don't send any message when analytics are switched off, so I don't know. 6. We need end-to-end encryption in Git, like keybase was providing, but open source and hostable in any git repo. I started a proof of concept of a custom remote where I just write all the git objects to another git repo, but haven't gotten very far. Disclaimer: I'm the author of GitJournal.

spicybright 4y ago
What's your opinion on SyncThing? I find this checks a ton of these boxes for me, and I find it really easy to get non-techies on board and actually be comfortable using it.
- nyanpasu64 4y ago
  Syncthing works well when devices can find each other (usually over mDNS) and are online simultaneously. After dealing with an entangled mess of introducers and devices, I decided to setup an old phone running Syncthing-Fork as root (otherwise mDNS doesn't work on Android), use it as a central introducer and always-online file host, and put my shared files there. It works pretty well, but I still run into cases sometimes where one client learns of the existence of files from another device, but the other device is offline and my central server doesn't have copies of them. For locally shared documents, I sync them with Syncthing and edit them on Android in Markor, and it works pretty well.
- bebna 4y ago
  I gone this route. I got different markdown editors on different devices and didn't want to use a public / cloud hosted git repo for my notes. It works fine, as long you don't get in contact with iOS. I got many devices and often let it sync my phone when I'm at home, because I'm struggling already enough with battery lifetime. I also got it running on one of my dedicated servers, so I got an offsite backup, with systemd you can set it on autostart after an internet connection was established, so I don't really have to manage it anymore and I only see it basically in the logs of the packages that I update.
  - spicybright 4y ago
    That's awesome. I use it pretty much the same, only I sync the root of my phone data too. So I can take a photo on my phone and have it pop up on my computer, ready to use. My only small tip for readers here is be careful treating any node as a safe backup. A device that accidentally deletes everything in a folder will propagate out if you don't notice it happened. But TBH in practice (as a very heavy user for my usecase) it's very rare, and a traditional backup system will cover you perfectly.

enasterosophes 4y ago

The Git protocol is free, and projects like libgit2 make it easy to integrate the protocol into new software. If you feel strongly about it, please consider contacting the purveyors of your favorite software and ask why they don't have Git integration. (Edit: libgit2, not libgit.)

deleted 4y ago
[deleted]

gernb 4y ago

sync to git isn't perfect. I use Google Keep, which, like docs, I can have the same document open on multiple machines, start editing on one, finished editing on the other, no "commit/push/fetch/rebase/merge" cycle required. The typical scenario for me is start typing something in and then get interrupted (notification, time based appointment, someone needing my attention), when I get back to entering I might be at a different machine (like started on my phone, got interrupted, then sitting at my computer and remember to finish). I don't see how git sync helps this issue.

oneeyedpigeon 4y ago
I don't think the OP was suggesting exposing all those low-level details in the UI. Surely the software would do all of that for you?

remram 4y ago

The Git model is to version everything together. This means that a conflict blocks the entire repo until it's fixed. In many situations, it's better to version separate pages/items/notes separately, and have a conflict to a single note leave the rest of your repo working and syncing.

YetAnotherNick 4y ago
Separate git repo for each note will fix this, right? Another possibility you can use some trick like https://stackoverflow.com/a/64810110/936584 https://stackoverflow.com/a/64810110/936584 to prefer one version.
- vhanda 4y ago
  This is similar to what GitJournal does. It'll never present you with a merge conflict and will instead just make prefer the local changes. It's not perfect, but in the worst case, the wrong option is chosen, but no data is lost since the history is there in Git.

codethief 4y ago

To be honest I'd rather love to see "select folder for storage" become a standard app feature. Then I could just use whatever app I want to synchronize the files (in my case probably Syncthing). Integration of $APP_FEATURES and $SYNC_BACKENDS in apps is a major waste of developer resources and it also means that, as a user, I can never freely choose the app based upon its intrinsic features but also always have to keep synchronization in mind.

mrtesthah 4y ago
The iOS editor Runestone takes this approach. I’ve installed Möbius Sync on the device, with both apps pointing at the same folder to create a syncthing conduit with my Mac. It’s fully seamless, private, and free.

dotancohen 4y ago

Git stores the entire history of text files. Very few apps need the history feature. Leaving the feature in place may take much disk space, and actually removing files no longer needed is a pain in git, git is not designed for that use case. I do agree that a standardized sync protocol and file format that could be pointed at any arbitrary server would be terrific. I just disagree that git is that format for many use cases.

prepend 4y ago
As a note taker, I’d much rather have the history and deal with the disk space than lose the history and have extra storage. I almost never truly want to delete any note. And most changes are additive so don’t take much extra storage due to versioning. I find it maddening to use a note app that doesn’t have detailed change logs and git’s history and blame is really useful.
- dotancohen 4y ago
  For notes, of course. I also store my notes in git. But GP mentioned "sync with git" as "a standard app feature", not specific to note apps.
vendiddy 4y ago
For text based notes, the storage cost is negligible. I've been using a single git repo to store notes for years. And my entire repo is a few MB.
- dotancohen 4y ago
  You just gave me an idea to garbage collect, let's see how much space we can save: $ du -sh ~/Notes/ 47M /home/dotancohen/Notes/ $ du -sh ~/Notes/.git 21M /home/dotancohen/Notes/.git $ git -C ~/Notes/.git gc $ du -sh ~/Notes/ 44M /home/dotancohen/Notes/
  - vhanda 4y ago
    It might be better to only measure the size of the `.git` folder, as with it you can reproduce everything else. Essentially, a bare git instance.

pydry 4y ago

I wish we just had save to textual flat file as a more common standard app feature (which is simple/easy) and a plethora of separate apps that did the syncing from that (hard - conflicts/authentication, but only needs to be done once for all apps). I have the git syncing tools but most apps dont save to flat file and the ones that do tend to require hitting export -> backup.

dominotw 4y ago

> Too many app developers realize they need some kind of shared state "in the cloud" and end up with a database and a backend server. This is usually the premium offering( eg: obsidian) in their product.