5 ms·

You can lose some file changes even after asking the OS to make sure they don't get lost, the normal way. That's a problem. It means e.g. transactional databas

You can lose some file changes even after asking the OS to make sure they don't get lost, the normal way. That's a problem. It means e.g. transactional databases (which cannot afford to lose data like that) have a huge performance hit on these machines, since they have to use F_FULLFSYNC. And since that "no really, save my data" feature is not the standard fsync(), it means any portable software compiled for Linux will be safe, but will be unsafe on macOS, by default. That is a significant gotcha. The question is why do other NVMe manufacturers not have such a performance penalty? 10x is fine; 1000x is not. This is something Apple should fix. It's a firmware problem.

EugeneOZ 5y ago
No, it’s not a problem, it is expected. If you are running a transactional database on your desktop - at least add a UPS to your system.
- olliej 5y ago
  The whole point of a transactional database is that even in the case of a power loss you do not lose data. If you UPS blows up, and so you lose power, you should not lose data. The point here is that on the apple systems if you do the correct thing your performance drops to that of spinning disks.
  - EugeneOZ 5y ago
    Add a secondary UPS. What will be the next excuse? It's ridiculous to think that in case of power loss you expect 100% data integrity - it might happen in the middle of the command execution. If the system should be unkillable, it should have an unkillable power source in the first place.
    - marcan_42 5y ago
      The entire point of modern journaling filesystems and properly designed transactional databases is to ensure 100% data integrity in case of power loss, every time, no matter what. The thinking you have is from the 1990s. We can (and do) do better today. A properly designed transactional database will only ever "fail ahead". If power fails a transaction that was in the process of committing might commit without an ack, but will never return an ack and then be lost on the next startup. The ack means the data is safe, regardless of what happened afterwards.
      - EugeneOZ 5y ago
        After reading this comment I’m not so sure about that: https://news.ycombinator.com/item?id=30371181 https://news.ycombinator.com/item?id=30371181
        marcan_42 5y ago
        That comment is about the semantics of OS APIs; filesystems are designed not to corrupt themselves in case of hard shutdown, and this is true as long as the underlying storage is well-behaved (e.g. honors flush requests). Databases on macOS already use F_FULLFSYNC (if they noticed this issue) to provide those guarantees. On Linux they just use fsync().
    - freemint 5y ago
      Since no Mac device has two power support adding a second UPS means chaining which will only increase the risk of something going wrong in the chain. Nobody expects 100% data integrity on power of. What is expected is that data that what was fsynced has 100% data integrity once that system call returns. This information is also used when moving files across the network, the file gets deleted on the sender when the receiver said fsync is completed. This means you could loose entire files of data when moving things over the network onto a Mac
      - EugeneOZ 5y ago
        This means the world would be full of complaints from macOS users, but for some reason, we only know about this detail because of that “shocking” Twitter thread.
        marcan_42 5y ago
        #!/usr/bin/python import os, sys, time, datetime t = datetime.datetime.now().isoformat() print(t) for i in range(5): time.sleep(1) print(4 - i) fd = os.open(sys.argv[1], os.O_RDWR|os.O_CREAT) os.lseek(fd, 0, 0) os.write(fd, b"test: " + t.encode("ascii") + b"\n"); os.fsync(fd) print("done!") time.sleep(100) Run that on a Mac Mini. Do it a couple times. Remember the timestamp of the last one. Let it count down, then pull the plug within a few seconds after "done!" shows up. Boot up again. The file contents will have reverted to a prior point in time. This isn't some hypothetical thing, this is a trivial test you can do. fsync() on macOS does not guarantee data is on stable storage. And this is actually well documented. Then if you want to see the performance problem, make it a loop instead and use `fcntl.fcntl(fd, fcntl.F_FULLFSYNC, 1)`. You'll get 40-odd IOPS, but at least your data won't disappear after power loss.
        EugeneOZ 5y ago
        Does it somehow prove that people are losing files when transferring them over the network? People would definitely notice it.
    - coldtea 5y ago
      >Add a secondary UPS. What will be the next excuse? You shouldn't have to add a secondary UPS at all, period, and still get that. Databases are designed that way (for integrity under sudden power loss) - the OS just needs to provide a standard call for the sync that they can use. Now, fsync not guaranteeing a write is one thing -- and it's common in other OSes, even Linux behaved like that. The non-commital fullsync on the other hand (and the slow speed) are problematic, and that's not an excuse for the user having such a bizarro case as wanting to run a DB on their Mac Mini without 2 UPS, that's you excusing Apple. Not to mention that 2 UPS wont solve the problem if you're not there to shut down the computer gracefully as they, themselves, are depleted (e.g. at night) when there's a powerloss.

EugeneOZ 5y ago

No, it’s not a problem, it is expected. If you are running a transactional database on your desktop - at least add a UPS to your system.

olliej 5y ago
The whole point of a transactional database is that even in the case of a power loss you do not lose data. If you UPS blows up, and so you lose power, you should not lose data. The point here is that on the apple systems if you do the correct thing your performance drops to that of spinning disks.
- EugeneOZ 5y ago
  Add a secondary UPS. What will be the next excuse? It's ridiculous to think that in case of power loss you expect 100% data integrity - it might happen in the middle of the command execution. If the system should be unkillable, it should have an unkillable power source in the first place.
  - marcan_42 5y ago
    The entire point of modern journaling filesystems and properly designed transactional databases is to ensure 100% data integrity in case of power loss, every time, no matter what. The thinking you have is from the 1990s. We can (and do) do better today. A properly designed transactional database will only ever "fail ahead". If power fails a transaction that was in the process of committing might commit without an ack, but will never return an ack and then be lost on the next startup. The ack means the data is safe, regardless of what happened afterwards.
    - EugeneOZ 5y ago
      After reading this comment I’m not so sure about that: https://news.ycombinator.com/item?id=30371181 https://news.ycombinator.com/item?id=30371181
      - marcan_42 5y ago
        That comment is about the semantics of OS APIs; filesystems are designed not to corrupt themselves in case of hard shutdown, and this is true as long as the underlying storage is well-behaved (e.g. honors flush requests). Databases on macOS already use F_FULLFSYNC (if they noticed this issue) to provide those guarantees. On Linux they just use fsync().
  - freemint 5y ago
    Since no Mac device has two power support adding a second UPS means chaining which will only increase the risk of something going wrong in the chain. Nobody expects 100% data integrity on power of. What is expected is that data that what was fsynced has 100% data integrity once that system call returns. This information is also used when moving files across the network, the file gets deleted on the sender when the receiver said fsync is completed. This means you could loose entire files of data when moving things over the network onto a Mac
    - EugeneOZ 5y ago
      This means the world would be full of complaints from macOS users, but for some reason, we only know about this detail because of that “shocking” Twitter thread.
      - marcan_42 5y ago
        #!/usr/bin/python import os, sys, time, datetime t = datetime.datetime.now().isoformat() print(t) for i in range(5): time.sleep(1) print(4 - i) fd = os.open(sys.argv[1], os.O_RDWR|os.O_CREAT) os.lseek(fd, 0, 0) os.write(fd, b"test: " + t.encode("ascii") + b"\n"); os.fsync(fd) print("done!") time.sleep(100) Run that on a Mac Mini. Do it a couple times. Remember the timestamp of the last one. Let it count down, then pull the plug within a few seconds after "done!" shows up. Boot up again. The file contents will have reverted to a prior point in time. This isn't some hypothetical thing, this is a trivial test you can do. fsync() on macOS does not guarantee data is on stable storage. And this is actually well documented. Then if you want to see the performance problem, make it a loop instead and use `fcntl.fcntl(fd, fcntl.F_FULLFSYNC, 1)`. You'll get 40-odd IOPS, but at least your data won't disappear after power loss.
        EugeneOZ 5y ago
        Does it somehow prove that people are losing files when transferring them over the network? People would definitely notice it.
  - coldtea 5y ago
    >Add a secondary UPS. What will be the next excuse? You shouldn't have to add a secondary UPS at all, period, and still get that. Databases are designed that way (for integrity under sudden power loss) - the OS just needs to provide a standard call for the sync that they can use. Now, fsync not guaranteeing a write is one thing -- and it's common in other OSes, even Linux behaved like that. The non-commital fullsync on the other hand (and the slow speed) are problematic, and that's not an excuse for the user having such a bizarro case as wanting to run a DB on their Mac Mini without 2 UPS, that's you excusing Apple. Not to mention that 2 UPS wont solve the problem if you're not there to shut down the computer gracefully as they, themselves, are depleted (e.g. at night) when there's a powerloss.