3 ms·
He talks about caching. As soon as you add caching, you now have a problem that isn't just close to syncing. It's utterly, absolutely identical to syncing. Sync
by api 11y ago
He talks about caching. As soon as you add caching, you now have a problem that isn't just close to syncing. It's utterly, absolutely identical to syncing. Syncing is just distributed caching where you only ever invalidate a cache entry when someone tells you to by deleting the local file.
You can "skip" the cache consistency problems by "just" always treating "the cloud" (which is internally probably a distributed cache) as the authority, but now you have an unreliable distributed system from a user's point of view since the thing you are skipping is how to handle real-world network weather. You can't eliminate problems by pretending they don't exist.
You also can't skip this problem with "the cloud." I am SO damn sick of "the cloud" being talked about as magic beans. If you're building a global distributed drive, then you are doing a distributed cache -- SYNCING -- in the cloud between cloud nodes. If you can do it there, why not do it on the end-user device too and let me cache my data? If you are using someone else's boxed cloud service to do this, then you are letting them build a distributed system and operate it for you in the cloud and if you ever have problems with it you better hope they have good support/SLA... and you're also doing something trivially duplicated. If that's what you're doing why don't I just use s3fs?
I sincerely hope there's more to it than this. I can trivially do this now with sshfs or Expandrive if I want a pretty GUI, and it's only tolerable with cloud storage that's physically very close to me (about <500mi).
- stuntprogrammer 11y agoI agree with many things you say. Take AWS S3 as an example: it reduces the problem by making objects immutable. Internally, they can cache aggressively without scaling fine-grained consistency on the objects as such. As an S3 user I can aggressively cache on the client too. Now we've reduced the problem to consistency on the metadata structure which aggregates objects for the user. There are "well known" ways to do this for traditional trees. Other well known options include doing an all search based approach (i.e. always talk to a server for metadata, perhaps with local result caching), and so on.
- api 11y agoThe last part you wrote is what I was getting at: you cannot eliminate distributed systems problems from distributed systems. You can only either (a) push them around into different parts of your design or (b) outsource them by using stuff like Amazon's DB-as-a-service stuff. Using immutable objects is (a): now you have a distributed metadata problem. If they're doing (a) then it's just another Dropbox but with less tolerance for disconnected operation. If they're doing (b) they're just reselling and maybe with a nice client. That's not doomed as a business of course but it's not all that technically interesting. If there's a (c) it's not in this article.
- stuntprogrammer 11y agoI strongly agree with you re 'conservation of distributed systems problems'. I'm being vague because I know what they're doing, but I'm limited in what I can comment on. Their public comments are that they are running their own servers and not reselling other storage. Personal opinion: no one can do (c) because of (a). That is, any possible (c) must tackle the fundamental hardness in the distributed systems problem, and this is what we agree (a) is doing. Using immutable objects as in S3 just shifts the problem elsewhere, while it reduces it, it doesn't solve it.
- api 11y agoImmutability does reduce the problem by reducing the data footprint of stuff that needs to be synchronized. So let me guess: They're doing a distributed metadata store where the cloud is the tie-breaker combined with flexible caching of immutable content-addressable (identified with SHA512 or similar) objects. In that case it sounds like cloud-hosted-only ipfs plus a nice client to access it from a host. Cold? Warm? :)