5 ms·

Do you have any good solutions for this exact use case yet?

imhoguy 6y ago
Checked Tahoe-LAFS, Perkeep, but they are too low level. Haven't tried Resilio Connect yet, although I would prefer something open-sourced. I think we have all the tech needed already (WebRTC, DHT, NAT hole punching, decades of p2p, encryption, onion security, StorJ/Filecoin) etc but what is lacking is dead simple UX and wide support of operating systems - Windows, Mac, Android, iOS, Linux (Raspberry, Synology, cheap VPS backup)
- newman314 6y ago
  I use Resilio Sync to sync my 1Password. It mostly works but is clunky in that the handling of identities is clumsy (unable to cleanly remove) as well as having less than stellar UX for adding/removing folders. Still looking for the perfect Dropbox-like experience but without the cloud sync piece. If only syncthing had decent mobile apps...
- hinkley 6y ago
  Yeah there are no turnkey solutions that I'm aware of. Which is why every time there's a thread like this I come to see what the rumpus is. I'm still waiting for a 'Drobo' like device without the proprietary physical layout. A light or dial goes into orange territory, you head to Best Buy or Amazon and buy the biggest drive that doesn't give you sticker shock, you push a button, out pops your worst drive and in goes the new one. Some lights flicker for a while and then go green. I thought ZFS would have given us almost everything but the hardware ten years ago, but it turned out they oversold a few of the features back then, and then Larry happened. Custom hardware is too expensive for small run consumer hardware, and Apple might have gotten into that space but never did. I wonder how many PCIe lanes you could shoehorn onto a Pi clone...
  - mceachen 6y ago
    Synology hardware is pretty close to what you're describing. Light on front goes from green to orange and you get an email, you plug in another drive, click a button in the GUI, and wait for the volume to reshard/resilver/remirror. A freenas box does this as well, but won't have the pretty drive light indicator if it's a home-built box, but then you're not limited to proprietary hardware.
    - hinkley 6y ago
      I had been meaning to look into Synology more and I watched a few reviews after this exchange. Sound good except I'm not happy about having to link to their servers. But everything works that way these days :/
      - thijsvandien 6y ago
        Why would you have to? Their devices work fine without any sort of account with them.
  - Reelin 6y ago
    4 TB drives are now commonly well under $100 (I've seen them as low as $70) and many Micro-ATX and NUC boards have 4+ SATA connectors. Btrfs has CoW, subvolumes, snapshots, and data integrity features. For the physical device at least, Debian and 2 Btrfs data drives in RAID 1 certainly isn't turnkey but seems quite accessible at this point.
- tekknolagi 6y ago
  Syncthing works pretty well here!
mceachen 6y ago
I'm hoping PhotoStructure solves this usecase when I add sharing in the next version. Note that PhotoStructure is focused on just photos and videos--it isn't a general file sharing solution. I'm looking into integrating with DAT or Hyperdrive-like solutions to help make storage backups less fiddly for users. For now, I recommend my beta users use SyncThing or Resilio Sync to get their photos and videos off their phones and on to their home NAS or computer.
Reelin 6y ago
Perhaps I'm completely misunderstanding something but don't centralized self hosted cloud-plus-app services address this exact usecase? Things like NextCloud, Seafile, and Sandstorm? The only major drawback seems to be that you have to host the physical hardware yourself due to lack of solid end-to-end encryption for most platforms. Sandstorm might have it (I'm not clear on what's client- and what's server- side there), Seafile has end-to-end encryption that doesn't protect metadata (https://forum.seafile.com/t/how-strong-is-the-encryption/6275 https://forum.seafile.com/t/how-strong-is-the-encryption/627...), and NextCloud appears to have a long-running beta of end-to-end encryption on a per-folder basis (https://nextcloud.com/endtoend/ https://nextcloud.com/endtoend/). Apparently Cryptomator exists (https://cryptomator.org/ https://cryptomator.org/) although I've never tried it myself. If you want to decentralize your shared files unfortunately last I checked SyncThing didn't yet support end-to-end encryption. (I don't think any of the other ones I mentioned can be used in a decentralized manner but things move quickly so I'm not sure.) Alternatively, if you were thinking more chat and messaging there's self hostable federated services such as PixelFed, PeerTube, Mastodon, and Matrix. Or was there some other usecase you had in mind?
- dpflug 6y ago
  SyncThing does support end-to-end encryption, and has for a while: https://docs.syncthing.net/users/security.html#sync-connections-bep https://docs.syncthing.net/users/security.html#sync-connecti...
  - Reelin 6y ago
    By end-to-end, I meant that the server can't see your data. For example, the idea of deploying one or more SyncThing instances to cloud providers that would never see your unencrypted data. There's an open GitHub issue about it since 2014 (https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109).

Checked Tahoe-LAFS, Perkeep, but they are too low level. Haven't tried Resilio Connect yet, although I would prefer something open-sourced. I think we have all the tech needed already (WebRTC, DHT, NAT hole punching, decades of p2p, encryption, onion security, StorJ/Filecoin) etc but what is lacking is dead simple UX and wide support of operating systems - Windows, Mac, Android, iOS, Linux (Raspberry, Synology, cheap VPS backup)

newman314 6y ago
I use Resilio Sync to sync my 1Password. It mostly works but is clunky in that the handling of identities is clumsy (unable to cleanly remove) as well as having less than stellar UX for adding/removing folders. Still looking for the perfect Dropbox-like experience but without the cloud sync piece. If only syncthing had decent mobile apps...
hinkley 6y ago
Yeah there are no turnkey solutions that I'm aware of. Which is why every time there's a thread like this I come to see what the rumpus is. I'm still waiting for a 'Drobo' like device without the proprietary physical layout. A light or dial goes into orange territory, you head to Best Buy or Amazon and buy the biggest drive that doesn't give you sticker shock, you push a button, out pops your worst drive and in goes the new one. Some lights flicker for a while and then go green. I thought ZFS would have given us almost everything but the hardware ten years ago, but it turned out they oversold a few of the features back then, and then Larry happened. Custom hardware is too expensive for small run consumer hardware, and Apple might have gotten into that space but never did. I wonder how many PCIe lanes you could shoehorn onto a Pi clone...
- mceachen 6y ago
  Synology hardware is pretty close to what you're describing. Light on front goes from green to orange and you get an email, you plug in another drive, click a button in the GUI, and wait for the volume to reshard/resilver/remirror. A freenas box does this as well, but won't have the pretty drive light indicator if it's a home-built box, but then you're not limited to proprietary hardware.
  - hinkley 6y ago
    I had been meaning to look into Synology more and I watched a few reviews after this exchange. Sound good except I'm not happy about having to link to their servers. But everything works that way these days :/
    - thijsvandien 6y ago
      Why would you have to? Their devices work fine without any sort of account with them.
- Reelin 6y ago
  4 TB drives are now commonly well under $100 (I've seen them as low as $70) and many Micro-ATX and NUC boards have 4+ SATA connectors. Btrfs has CoW, subvolumes, snapshots, and data integrity features. For the physical device at least, Debian and 2 Btrfs data drives in RAID 1 certainly isn't turnkey but seems quite accessible at this point.
tekknolagi 6y ago
Syncthing works pretty well here!

mceachen 6y ago

I'm hoping PhotoStructure solves this usecase when I add sharing in the next version. Note that PhotoStructure is focused on just photos and videos--it isn't a general file sharing solution. I'm looking into integrating with DAT or Hyperdrive-like solutions to help make storage backups less fiddly for users. For now, I recommend my beta users use SyncThing or Resilio Sync to get their photos and videos off their phones and on to their home NAS or computer.

Reelin 6y ago

Perhaps I'm completely misunderstanding something but don't centralized self hosted cloud-plus-app services address this exact usecase? Things like NextCloud, Seafile, and Sandstorm? The only major drawback seems to be that you have to host the physical hardware yourself due to lack of solid end-to-end encryption for most platforms. Sandstorm might have it (I'm not clear on what's client- and what's server- side there), Seafile has end-to-end encryption that doesn't protect metadata (https://forum.seafile.com/t/how-strong-is-the-encryption/6275 https://forum.seafile.com/t/how-strong-is-the-encryption/627...), and NextCloud appears to have a long-running beta of end-to-end encryption on a per-folder basis (https://nextcloud.com/endtoend/ https://nextcloud.com/endtoend/). Apparently Cryptomator exists (https://cryptomator.org/ https://cryptomator.org/) although I've never tried it myself. If you want to decentralize your shared files unfortunately last I checked SyncThing didn't yet support end-to-end encryption. (I don't think any of the other ones I mentioned can be used in a decentralized manner but things move quickly so I'm not sure.) Alternatively, if you were thinking more chat and messaging there's self hostable federated services such as PixelFed, PeerTube, Mastodon, and Matrix. Or was there some other usecase you had in mind?

dpflug 6y ago
SyncThing does support end-to-end encryption, and has for a while: https://docs.syncthing.net/users/security.html#sync-connections-bep https://docs.syncthing.net/users/security.html#sync-connecti...
- Reelin 6y ago
  By end-to-end, I meant that the server can't see your data. For example, the idea of deploying one or more SyncThing instances to cloud providers that would never see your unencrypted data. There's an open GitHub issue about it since 2014 (https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109).