5 ms·

Posix Compatibility Comparison: GCP Filestore, Amazon EFS, and Azure Files

45 points by jinqueeny 4y ago

pjmlp 4y ago
Anyone that expects UNIX semantics on native cloud workloads, or mobile phones, is in for a few surprises. "Transcending POSIX: The End of an Era?" https://www.usenix.org/publications/loginonline/transcending-posix-end-era https://www.usenix.org/publications/loginonline/transcending...
- actionfromafar 4y ago
  Yep. The developers truly are holding it wrong.
jiggawatts 4y ago
Having worked with Azure for a number of years now, I get the distinct impression that it is developed by people banging out features because they are told to do so, and have no interest in what the feature is for, who uses it, and how. There is nearly zero quality assurance, because angry customers will shout at support. Support is not allowed to discuss this with the developers, instead they will tell customers to "suggest" features -- such as POSIX compatibility -- on a public feedback forum and then this gets voted up, like a popularity contest. Imagine going to a hospital run like this, where the surgeons just do whatever they're told ("I got told to amputate, so I did. What do you mean check which leg to cut off?"), have zero interest in basic hygiene let alone antiseptic practices, and then angry patients can submit a piece of paper into a suggestion box with requests like: "Please cut the correct leg off! Also, it would be nice not to get a massive infection after surgery." To which there will be a curt reply saying: "Customers can scale to additional legs, and they can purchase their own antibiotics if they so choose. Closed, will-not-fix."
- encryptluks2 4y ago
  Microsoft notoriously ignores feedback, even those with the highest votes.
  - saurik 4y ago
    Do you feel this is more true than with Google? Maybe the difference is that we care less, because Google cared enough to get the feature to work correctly? ;P
    - encryptluks2 4y ago
      At least for Chromium and Android, they are open source and more standard to propose and contribute changes. Yes, Google isn't perfect but they have done a much better job overall for their consumer tools and products.
- Arnavion 4y ago
  Weird take to equate a commercial cloud with a life-saving hospital. Doctors have a duty of care to patients. Nobody's going to die because Azure Splines didn't implement the reticulation feature that people are asking for in their forums.
  - jiggawatts 4y ago
    A significant chunk of our state’s Department of Health is hosted on Azure. An outage could result in deaths, or reduced standard of care.
    - jchw 4y ago
      I hope this isn't a ridiculous take, but even if Azure or any other cloud solution is used in healthcare, I would strongly hope that it isn't put in a position so critical to the point where it would literally lead to deaths if it were down for a while.
      - actionfromafar 4y ago
        What position would that be? Anything at a hospital can lead to death if inefficient or down.
        jiggawatts 4y ago
        Not to mention that Azure is specifically targeting the health industry with offerings like confidential compute.
    - Arnavion 4y ago
      Why are you changing the topic? We were talking about features, not outages.
      - jiggawatts 4y ago
        A lack of POSIX compliance generally means a lack of data integrity and basic correctness. In a file system that could result in data loss, data corruption, and the like.
- bushbaba 4y ago
  Having used AWS, gcp, and azure. It’s clear AWS has a decade lead. It’s not just the technology, but Amazon’s relentless customer obsession and culture that drives a better platform.
- notTooFarGone 4y ago
  It really seems that way. We are using a bunch of microsoft azure "maintained" stuff, but they don't update their products and seemingly noone is making sure that shit works intended. This results in us building our own stuff and defeating the purpose entirely.
- mardifoufs 4y ago
  Oh my god, yes. I used to like it, but once you dive just a bit deeper into literally any feature, you will always always find quirks that make absolutely no sense. There is always some sort of limitation that is obviously caused by shipping anything they can as fast as possible. Absolutely 0 polish. A small but frustrating example I experienced just today: In Azure ML, you can create data assets (basically a sort of centeal syslink to your actual data in a storage container) so that you can version your data and access it easily in your pipelines. Sounds great! But they recently pushed a new version that makes it impossible to create a data asset with multiple folders (!). You used to be able to do it, but now your data asset can either be a single file, a single folder (with no subfolders!), or a tabular asset. It's completely insane to me because you now need to create a data asset for every folder you have. on a product made for data-heavy ML workloads. If you have data across a few dozens/hundreds of folders, you can't aggregate them even if they are to be used on the same pipeline, unless you do it manually. Also, now that you have dozens of unnecessary data assets, you can't even set your pipeline to take an arbitrary amount of them as input. You have 20 data assets/folders? Have fun adding 20 inputs in your component's YAML. I can get around it, and maybe I'm dumb and all of our data should have been in a single folder anyway. But it used to work, with no downsides! This stuff happens every day and can be especially aggravating in "heavier" services like AKS.
klaas- 4y ago
it looks like they are comparing the cifs/samba Azure Files (https://learn.microsoft.com/en-us/azure/storage/files/files-smb-protocol?tabs=azure-portal https://learn.microsoft.com/en-us/azure/storage/files/files-...) service instead of the nfs (files - https://learn.microsoft.com/en-us/azure/storage/files/files-nfs-protocol https://learn.microsoft.com/en-us/azure/storage/files/files-...) / sftp (blob - https://learn.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support https://learn.microsoft.com/en-us/azure/storage/blobs/secure...) services which offer more posix compliance
- daviesliu 4y ago
  We had compared nfs of Azure in the article, I believe cifs/samba should be much worse.
  - klaas- 4y ago
    From your article "For example, Azure Files files and folders have default permissions of 0777, with root as owner, and it does not support modification" that sounds like cifs, I can't tell you much about Azure Files NFS but I do know they support basic file rights and setting of uid/gid, my company uses them in Azure for SAP.
    - klaas- 4y ago
      one more thing, there are two NFS implementations, one v3 and one v4; the v3 is on top of blobs, the v4 one is on top of azure files. we use the v4 one, never tried the blob one :)
      - daviesliu 4y ago
        Yes, we picked the default one in docs, which should be NFS v3, will redo the test against NFS v4 and update the article, thanks!
    - daviesliu 4y ago
      Azure has Azure Files and Azrue NetApp Files, the later one is provided from NetApp. Azure Files was used in the article, maybe you are using NetApp Files? We will update the article to make it clear, thanks!
dekhn 4y ago
POSIX compatibility requirements are an antipattern.
908B64B197 4y ago
Honest question here: does anyone care about POSIX anymore? This isn't the 90's anymore. I don't think any of the major operating systems broadly in use today even claim to be compatible with it. Now, ignoring the weird grammar of the article, I'm not sure what it's comparing considering it's looking at three different products each specifically claiming to support a different feature set. It's doubly confusing for Azure since they do have a CIFS/SMB offering as well as two different implementations of NFS and the author doesn't reveal which one he picked (of course samba won't map 1:1 to all features a *NIX system expects). To me it looks like clickbait. The company behind this also pushed another marketing piece on the frontpage today https://news.ycombinator.com/item?id=34706227 https://news.ycombinator.com/item?id=34706227

45 points by jinqueeny 4y ago

Anyone that expects UNIX semantics on native cloud workloads, or mobile phones, is in for a few surprises. "Transcending POSIX: The End of an Era?" https://www.usenix.org/publications/loginonline/transcending-posix-end-era https://www.usenix.org/publications/loginonline/transcending...

actionfromafar 4y ago
Yep. The developers truly are holding it wrong.

jiggawatts 4y ago

Having worked with Azure for a number of years now, I get the distinct impression that it is developed by people banging out features because they are told to do so, and have no interest in what the feature is for, who uses it, and how. There is nearly zero quality assurance, because angry customers will shout at support. Support is not allowed to discuss this with the developers, instead they will tell customers to "suggest" features -- such as POSIX compatibility -- on a public feedback forum and then this gets voted up, like a popularity contest. Imagine going to a hospital run like this, where the surgeons just do whatever they're told ("I got told to amputate, so I did. What do you mean check which leg to cut off?"), have zero interest in basic hygiene let alone antiseptic practices, and then angry patients can submit a piece of paper into a suggestion box with requests like: "Please cut the correct leg off! Also, it would be nice not to get a massive infection after surgery." To which there will be a curt reply saying: "Customers can scale to additional legs, and they can purchase their own antibiotics if they so choose. Closed, will-not-fix."

encryptluks2 4y ago
Microsoft notoriously ignores feedback, even those with the highest votes.
- saurik 4y ago
  Do you feel this is more true than with Google? Maybe the difference is that we care less, because Google cared enough to get the feature to work correctly? ;P
  - encryptluks2 4y ago
    At least for Chromium and Android, they are open source and more standard to propose and contribute changes. Yes, Google isn't perfect but they have done a much better job overall for their consumer tools and products.
Arnavion 4y ago
Weird take to equate a commercial cloud with a life-saving hospital. Doctors have a duty of care to patients. Nobody's going to die because Azure Splines didn't implement the reticulation feature that people are asking for in their forums.
- jiggawatts 4y ago
  A significant chunk of our state’s Department of Health is hosted on Azure. An outage could result in deaths, or reduced standard of care.
  - jchw 4y ago
    I hope this isn't a ridiculous take, but even if Azure or any other cloud solution is used in healthcare, I would strongly hope that it isn't put in a position so critical to the point where it would literally lead to deaths if it were down for a while.
    - actionfromafar 4y ago
      What position would that be? Anything at a hospital can lead to death if inefficient or down.
      - jiggawatts 4y ago
        Not to mention that Azure is specifically targeting the health industry with offerings like confidential compute.
  - Arnavion 4y ago
    Why are you changing the topic? We were talking about features, not outages.
    - jiggawatts 4y ago
      A lack of POSIX compliance generally means a lack of data integrity and basic correctness. In a file system that could result in data loss, data corruption, and the like.
bushbaba 4y ago
Having used AWS, gcp, and azure. It’s clear AWS has a decade lead. It’s not just the technology, but Amazon’s relentless customer obsession and culture that drives a better platform.
notTooFarGone 4y ago
It really seems that way. We are using a bunch of microsoft azure "maintained" stuff, but they don't update their products and seemingly noone is making sure that shit works intended. This results in us building our own stuff and defeating the purpose entirely.
mardifoufs 4y ago
Oh my god, yes. I used to like it, but once you dive just a bit deeper into literally any feature, you will always always find quirks that make absolutely no sense. There is always some sort of limitation that is obviously caused by shipping anything they can as fast as possible. Absolutely 0 polish. A small but frustrating example I experienced just today: In Azure ML, you can create data assets (basically a sort of centeal syslink to your actual data in a storage container) so that you can version your data and access it easily in your pipelines. Sounds great! But they recently pushed a new version that makes it impossible to create a data asset with multiple folders (!). You used to be able to do it, but now your data asset can either be a single file, a single folder (with no subfolders!), or a tabular asset. It's completely insane to me because you now need to create a data asset for every folder you have. on a product made for data-heavy ML workloads. If you have data across a few dozens/hundreds of folders, you can't aggregate them even if they are to be used on the same pipeline, unless you do it manually. Also, now that you have dozens of unnecessary data assets, you can't even set your pipeline to take an arbitrary amount of them as input. You have 20 data assets/folders? Have fun adding 20 inputs in your component's YAML. I can get around it, and maybe I'm dumb and all of our data should have been in a single folder anyway. But it used to work, with no downsides! This stuff happens every day and can be especially aggravating in "heavier" services like AKS.

klaas- 4y ago

it looks like they are comparing the cifs/samba Azure Files (https://learn.microsoft.com/en-us/azure/storage/files/files-smb-protocol?tabs=azure-portal https://learn.microsoft.com/en-us/azure/storage/files/files-...) service instead of the nfs (files - https://learn.microsoft.com/en-us/azure/storage/files/files-nfs-protocol https://learn.microsoft.com/en-us/azure/storage/files/files-...) / sftp (blob - https://learn.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support https://learn.microsoft.com/en-us/azure/storage/blobs/secure...) services which offer more posix compliance

daviesliu 4y ago
We had compared nfs of Azure in the article, I believe cifs/samba should be much worse.
- klaas- 4y ago
  From your article "For example, Azure Files files and folders have default permissions of 0777, with root as owner, and it does not support modification" that sounds like cifs, I can't tell you much about Azure Files NFS but I do know they support basic file rights and setting of uid/gid, my company uses them in Azure for SAP.
  - klaas- 4y ago
    one more thing, there are two NFS implementations, one v3 and one v4; the v3 is on top of blobs, the v4 one is on top of azure files. we use the v4 one, never tried the blob one :)
    - daviesliu 4y ago
      Yes, we picked the default one in docs, which should be NFS v3, will redo the test against NFS v4 and update the article, thanks!
  - daviesliu 4y ago
    Azure has Azure Files and Azrue NetApp Files, the later one is provided from NetApp. Azure Files was used in the article, maybe you are using NetApp Files? We will update the article to make it clear, thanks!

dekhn 4y ago

POSIX compatibility requirements are an antipattern.

908B64B197 4y ago

Honest question here: does anyone care about POSIX anymore? This isn't the 90's anymore. I don't think any of the major operating systems broadly in use today even claim to be compatible with it. Now, ignoring the weird grammar of the article, I'm not sure what it's comparing considering it's looking at three different products each specifically claiming to support a different feature set. It's doubly confusing for Azure since they do have a CIFS/SMB offering as well as two different implementations of NFS and the author doesn't reveal which one he picked (of course samba won't map 1:1 to all features a *NIX system expects). To me it looks like clickbait. The company behind this also pushed another marketing piece on the frontpage today https://news.ycombinator.com/item?id=34706227 https://news.ycombinator.com/item?id=34706227