3 ms·

NFS? I just simply don't see the place for this product. The world moved towards the use of smart data formats on platforms like S3 and HDFS. Where would be a N

• notacoward 12y ago
  There are two things wrong with the "nobody should use a distributed file system" meme. (Disclaimer: I'm a distributed file system developer.) (1) Not all development is green-field and thus subject to using this week's fashionable data store. A lot of applications already use file systems and rely on their semantics. Ripping up the entire storage layer of a large and complex application would be irresponsible, as it's likely to create more problems than it solves. Look down your nose at "legacy code" all you want, but those users exist and have money to give people who will help them solve their business problems. Often the solutions are as cutting-edge as anything you'd find in more fashionable quarters, even if the facade is old fashioned. (2) Even for green-field development, file systems are often a better fit than object stores. Nested directories are more than just an organizational scheme, but also support security in a way that object stores don't. What's the equivalent of SELinux for data stored in S3 or Swift? There is none. File system consistency/durability guarantees can be important, as can single-byte writes to the middle of a file, while parsing HTTP headers on every request drops a lot of functionality and performance on the floor. Distributed databases are much more compelling, but the fact remains that the file system model is often the best one even for new applications. Go ahead and use something else if it suits you. Other people wouldn't have much use for your favorite storage model, and this one suits them perfectly.
  • istvan__ 12y ago
    Funny you mentioned that this is a meme to you, while it is really a technical consideration to me, and I supplied some details about my concerns. Answers following your numbering: (1) Calling S3 "this week's fashionable data store" is like saying that an elephant is an interesting microbe. The rest of your points about "please do not innovate, we have filesystems for 40 years and this is how you store your data". I do not agree. Disclaimer, I was member of the team that moved amazon.com from using an NFS based storage to S3. It was a great success, and it solved many of our problems including dealing with the insane amount of issues that was introduced by running an NFS cluster at that scale. And I would like to emphasize on scale, because your operational problems are quite often increase worse than linear with you scale. I know about legacy code, and running several legacy services in production as of now. I can tell you one thing. There is a point when it is not financially viable to keep rolling with the legacy code. This point is very different based on your actual use case, banks tend to run "legacy code" while web2.0 companies tend to innovate and replace systems with a faster peace. I don't see any conflict here. We even did a compatibility layer for the new solution and it was possible to run your legacy code using the new system and your software was untouched. (2) Nested directories is a logical layer on the top of how the data is stored, aka a view, your are a distributed FS developer so I guess you understand that. S3 also supports nested directories no biggie here. Security. Well this is kind of weird because last time I checked S3 had an extensive security http://aws.amazon.com/s3/faqs/#security_anchor http://aws.amazon.com/s3/faqs/#security_anchor Now the rest of your question can be re-phrased: "I am used to X why isn't there X with this new thing???" I am not sure how many of the file system users use SElinux my educated guess is that roughly around 1-10%. It is a very complex system that not too many companies invest using. For our use cases the fine grained ACLs were good enough so we are using those. File system durability: yes it is very important, this why I was kind of shocked about this bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/45 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/317781/... I guess you are right about the overhead of reading and writing, dealing with http headers etc. If the systems that benefit the most from S3 where single node systems it would be silly to use S3 at the first place. We are talking about 1000 - 10000 computers using the same data storage layer. And you can tell me if I am wrong but if you would like to access the same files on these nodes using a FS than you are going to end up with a locking hell. This is why modern software that is IO heavy moved away from in-place edits towards the "lock free" data access. Look at the implementation of Kafka log files or how Aeron writes files. This is exactly the same schematics how use use S3. Accident? ;) I would like to repeat my original question: I don't see huge market for a distributed FS. I might be wrong, but this is how I see it. http://kafka.apache.org/ http://kafka.apache.org/ https://github.com/real-logic/Aeron https://github.com/real-logic/Aeron