8 ms·

I'm always glad to see people experimenting with different approaches to building and deploying apps. That said, the general idea of this Serverfree approach do

by jimmar 3y ago

I'm always glad to see people experimenting with different approaches to building and deploying apps. That said, the general idea of this Serverfree approach doesn't appeal to me. On any given day, I'll use 4 different devices. I need my data to synchronize seamlessly. I wouldn't use a program (in-browser or traditionally installed application) unless I can synchronize that data either by storing files in a Dropbox-like program or in the cloud. I don't want to have to remember which computer/browser combination I was working on. Edit: forgot some words

gamepsys 3y ago
I often think we should have a separation of concern between storing/syncing data and application code. That syncing app files should be an OS level feature, with many different available data stores, and sync well across different platforms.
- xmprt 3y ago
  Conflict resolution logic is typically application specific so it would be hard to get something like this working in practice.
  - simultsop 3y ago
    If it is a solo user, latest data would be enough to sync. Asuming the blog post tackles personal need app, single user.
    - LtWorf 3y ago
      What if you were offline for a while?
  - vbezhenar 3y ago
    Do not allow offline mode and make fsync block until server acknowledges write.
    - _yb2s 3y ago
      Depending on the nature of the data, you can design the app to have a write only journal with independent entries, that are always possible to merge in a consistent way at a later time. This obviously works better for very simple things like a "play history" and not for complex things like collaborative document writing.
  - TheCoreh 3y ago
    iCloud already supports this, and from what I gather it works decently well: https://developer.apple.com/documentation/uikit/uidocumentstate/uidocumentstateinconflict https://developer.apple.com/documentation/uikit/uidocumentst... https://developer.apple.com/library/archive/technotes/tn2336/_index.html https://developer.apple.com/library/archive/technotes/tn2336... Essentially there's an OS mechanism for getting the different versions of the file, your app can detect conflicts and choose how to resolve them, including by displaying arbitrary UI to the user. IIRC This UI can even be integrated with the file open dialog. As a last resort fallback if the app doesn't resolve it Finder/Files will let you keep either version or both (as separate files)
    - jeremyjh 3y ago
      > your app can detect conflicts and choose how to resolve Like GP said - app specific. If I have two different versions of a SQLite database, that started from a common revision but were both updated independently, and want the updates that were applied in each of them to be preserved - there is quite a lot of work left to do after Apple throws their hands up.
      - saurik 3y ago
        You wouldn't (and don't) use this kind of mechanism at the file-level on an entire SQLite database: just because someone suggests something be an OS feature doesn't mean that existing software would work without changes.
        jeremyjh 3y ago
        > You wouldn't (and don't) use this kind of mechanism at the file-level on an entire SQLite database: But the OFA is about databases, and this thread is about syncing databases. So you agree with me: the Apple feature is worthless for this use case. GGP was right that its a lot of work to build sync with conflict resolution into an application, and its application specifc.
- jasonjayr 3y ago
  It should be OS-level, but in 2024 NO vendor (Apple, Microsoft, Google) is going to make a new open protocol for cross-platform syncing, w/o binding it tightly to their authentication platform/device security system/etc.
simultsop 3y ago
Exactly, initially we try to ditch data storage, lets say we do it by encrypting db and syncing to a cloud service, then expect it syncs on other devices when we're about to use the app. This would be just CRUD. Background processes would be a whole different problem to tackle with. I still can't envision a self-hosted decentralized backend on trivial devices.
- aspenmayer 3y ago
  I think that’s what Veilid aspires to be. Have you looked into it? If so, please let me know what you think about it in this context. https://en.wikipedia.org/wiki/Veilid https://en.wikipedia.org/wiki/Veilid > Veilid is a peer-to-peer network and application framework released by the Cult of the Dead Cow on August 11, 2023, at DEF CON 31. Described by its authors as "like Tor, but for apps", it is written in Rust, and runs on Linux, macOS, Windows, Android, iOS, and in-browser WASM. VeilidChat is a secure messaging application built on Veilid. https://youtube.com/watch?v=Kb1lKscAMDQ https://youtube.com/watch?v=Kb1lKscAMDQ
_yb2s 3y ago
This can be done through a shared database file that syncs however you want- e.g. stored in a shared cloud, etc. The app itself then needs to have very robust conflict resolution code, but it can be done. I think some of the more security focused open source password manager apps already use an approach like this.
pylua 3y ago
Not to mention what if I spill my coffee all over my laptop or otherwise lose access
Shorel 3y ago
Same here, Every single product or service I pay for, needs to work on Linux and on Android. Like Spotify. I love that the music plays on Linux with the big speakers, but I am choosing the songs on the Android pocket device.
- ruslan_talpa 3y ago
  since this is basically running in a (modern) browser, doesn't it follow that it works on all platforms where there is a browser?
  - Shorel 3y ago
    It was only an example, and in both platforms I use an installed application. A more interesting example: I use pCloud instead of MS OneDrive, or Google Drive. They don't have supported Linux clients. I rejected Dropbox for other reasons, like the limitation to have only one synchronized folder, but technically it has an Ubuntu service and an Android app. Icedrive seems to be a suitable service, but I found pCloud first. And believe me: I will not use a browser based interface when I could simply save the files into several folders of my preference, and the sync happens in the background. Every single file storage service I mentioned has a browser based interface. I consider all of them unusable.
ruslan_talpa 3y ago
I was just made aware TursoDB has a version of their client [1] with the same underlying technology so it seems it might be possible to have the best of both worlds, interacting locally with the db in the browser while it's being synced to the remote instance (not 100% sure though). 1. https://www.npmjs.com/package/@libsql/client-wasm https://www.npmjs.com/package/@libsql/client-wasm
layer8 3y ago
For me the sweet spot are applications that store data on the local file system, but where the data is actually synchronized using something like NextCloud or Dropbox (optionally encrypted with something like Cryptomator) or iCloud, and the applications are built to support the synchronization by merging changes from different devices and detecting and resolving conflicts. Meaning, the only cloud component should be “dumb” data storage, and it should remain entirely optional, only needed for use across multiple devices.
hosh 3y ago
There are private, distributed, synchronization protocols people are working on, such as Willow Protocol. They are still working things out. But synchronization across devices controlled by a principal is doable with the primitives they have already come up with.
samwillis 3y ago
There are a lot of exciting sync technologies being develop for this use case, I work on one of them at ElectricSQL (mentioned at the end by the OP), but we maintain a list of alternative here: https://electric-sql.com/docs/reference/alternatives https://electric-sql.com/docs/reference/alternatives
ochiba 3y ago
I agree, and the other downside is that with the server-free approach described in the article, there wouldn't be a backup of your data off your device. The author does mention privacy concerns — hence the appeal of storing the data locally on your device. I work on PowerSync https://www.powersync.com/ https://www.powersync.com/ — using embedded SQLite for local-first/offline-first which syncs with Postgres in the background. I think using an architecture like that where an encrypted version of the data is synced to Postgres, and decrypted for access on the client, would balance the trade-offs well.
ssss11 3y ago
Is there anything stopping you from also syncing an encrypted copy of your data to a central server to be synced down or your other devices? It doesn’t appear so. A websocket between all your devices and a server (or even webrtc between devices) could achieve this in parallel.
- Tactician_mark 3y ago
  I was going to mention WebRTC! It seems designed for video calling, but there are lots of cool use cases - I recently ran across https://github.com/dmotz/trystero https://github.com/dmotz/trystero , a dead simple WebRTC library for peer-to-peer multiplayer browser games.

by jimmar 3y ago

gamepsys 3y ago

I often think we should have a separation of concern between storing/syncing data and application code. That syncing app files should be an OS level feature, with many different available data stores, and sync well across different platforms.

xmprt 3y ago
Conflict resolution logic is typically application specific so it would be hard to get something like this working in practice.
- simultsop 3y ago
  If it is a solo user, latest data would be enough to sync. Asuming the blog post tackles personal need app, single user.
  - LtWorf 3y ago
    What if you were offline for a while?
- vbezhenar 3y ago
  Do not allow offline mode and make fsync block until server acknowledges write.
  - _yb2s 3y ago
    Depending on the nature of the data, you can design the app to have a write only journal with independent entries, that are always possible to merge in a consistent way at a later time. This obviously works better for very simple things like a "play history" and not for complex things like collaborative document writing.
- TheCoreh 3y ago
  iCloud already supports this, and from what I gather it works decently well: https://developer.apple.com/documentation/uikit/uidocumentstate/uidocumentstateinconflict https://developer.apple.com/documentation/uikit/uidocumentst... https://developer.apple.com/library/archive/technotes/tn2336/_index.html https://developer.apple.com/library/archive/technotes/tn2336... Essentially there's an OS mechanism for getting the different versions of the file, your app can detect conflicts and choose how to resolve them, including by displaying arbitrary UI to the user. IIRC This UI can even be integrated with the file open dialog. As a last resort fallback if the app doesn't resolve it Finder/Files will let you keep either version or both (as separate files)
  - jeremyjh 3y ago
    > your app can detect conflicts and choose how to resolve Like GP said - app specific. If I have two different versions of a SQLite database, that started from a common revision but were both updated independently, and want the updates that were applied in each of them to be preserved - there is quite a lot of work left to do after Apple throws their hands up.
    - saurik 3y ago
      You wouldn't (and don't) use this kind of mechanism at the file-level on an entire SQLite database: just because someone suggests something be an OS feature doesn't mean that existing software would work without changes.
      - jeremyjh 3y ago
        > You wouldn't (and don't) use this kind of mechanism at the file-level on an entire SQLite database: But the OFA is about databases, and this thread is about syncing databases. So you agree with me: the Apple feature is worthless for this use case. GGP was right that its a lot of work to build sync with conflict resolution into an application, and its application specifc.
jasonjayr 3y ago
It should be OS-level, but in 2024 NO vendor (Apple, Microsoft, Google) is going to make a new open protocol for cross-platform syncing, w/o binding it tightly to their authentication platform/device security system/etc.

simultsop 3y ago

Exactly, initially we try to ditch data storage, lets say we do it by encrypting db and syncing to a cloud service, then expect it syncs on other devices when we're about to use the app. This would be just CRUD. Background processes would be a whole different problem to tackle with. I still can't envision a self-hosted decentralized backend on trivial devices.

aspenmayer 3y ago
I think that’s what Veilid aspires to be. Have you looked into it? If so, please let me know what you think about it in this context. https://en.wikipedia.org/wiki/Veilid https://en.wikipedia.org/wiki/Veilid > Veilid is a peer-to-peer network and application framework released by the Cult of the Dead Cow on August 11, 2023, at DEF CON 31. Described by its authors as "like Tor, but for apps", it is written in Rust, and runs on Linux, macOS, Windows, Android, iOS, and in-browser WASM. VeilidChat is a secure messaging application built on Veilid. https://youtube.com/watch?v=Kb1lKscAMDQ https://youtube.com/watch?v=Kb1lKscAMDQ

_yb2s 3y ago

This can be done through a shared database file that syncs however you want- e.g. stored in a shared cloud, etc. The app itself then needs to have very robust conflict resolution code, but it can be done. I think some of the more security focused open source password manager apps already use an approach like this.

pylua 3y ago

Not to mention what if I spill my coffee all over my laptop or otherwise lose access

Shorel 3y ago

Same here, Every single product or service I pay for, needs to work on Linux and on Android. Like Spotify. I love that the music plays on Linux with the big speakers, but I am choosing the songs on the Android pocket device.

ruslan_talpa 3y ago
since this is basically running in a (modern) browser, doesn't it follow that it works on all platforms where there is a browser?
- Shorel 3y ago
  It was only an example, and in both platforms I use an installed application. A more interesting example: I use pCloud instead of MS OneDrive, or Google Drive. They don't have supported Linux clients. I rejected Dropbox for other reasons, like the limitation to have only one synchronized folder, but technically it has an Ubuntu service and an Android app. Icedrive seems to be a suitable service, but I found pCloud first. And believe me: I will not use a browser based interface when I could simply save the files into several folders of my preference, and the sync happens in the background. Every single file storage service I mentioned has a browser based interface. I consider all of them unusable.

ruslan_talpa 3y ago

I was just made aware TursoDB has a version of their client [1] with the same underlying technology so it seems it might be possible to have the best of both worlds, interacting locally with the db in the browser while it's being synced to the remote instance (not 100% sure though). 1. https://www.npmjs.com/package/@libsql/client-wasm https://www.npmjs.com/package/@libsql/client-wasm

layer8 3y ago

For me the sweet spot are applications that store data on the local file system, but where the data is actually synchronized using something like NextCloud or Dropbox (optionally encrypted with something like Cryptomator) or iCloud, and the applications are built to support the synchronization by merging changes from different devices and detecting and resolving conflicts. Meaning, the only cloud component should be “dumb” data storage, and it should remain entirely optional, only needed for use across multiple devices.

hosh 3y ago

There are private, distributed, synchronization protocols people are working on, such as Willow Protocol. They are still working things out. But synchronization across devices controlled by a principal is doable with the primitives they have already come up with.

samwillis 3y ago

There are a lot of exciting sync technologies being develop for this use case, I work on one of them at ElectricSQL (mentioned at the end by the OP), but we maintain a list of alternative here: https://electric-sql.com/docs/reference/alternatives https://electric-sql.com/docs/reference/alternatives

ochiba 3y ago

I agree, and the other downside is that with the server-free approach described in the article, there wouldn't be a backup of your data off your device. The author does mention privacy concerns — hence the appeal of storing the data locally on your device. I work on PowerSync https://www.powersync.com/ https://www.powersync.com/ — using embedded SQLite for local-first/offline-first which syncs with Postgres in the background. I think using an architecture like that where an encrypted version of the data is synced to Postgres, and decrypted for access on the client, would balance the trade-offs well.

ssss11 3y ago

Is there anything stopping you from also syncing an encrypted copy of your data to a central server to be synced down or your other devices? It doesn’t appear so. A websocket between all your devices and a server (or even webrtc between devices) could achieve this in parallel.

Tactician_mark 3y ago
I was going to mention WebRTC! It seems designed for video calling, but there are lots of cool use cases - I recently ran across https://github.com/dmotz/trystero https://github.com/dmotz/trystero , a dead simple WebRTC library for peer-to-peer multiplayer browser games.