4 ms·

As a laptop user I would probably opt to make the same choice as Apple here. I like the idea mentioned to allow a tunable parameter to only allow ever losing 1

As a laptop user I would probably opt to make the same choice as Apple here. I like the idea mentioned to allow a tunable parameter to only allow ever losing 1 second of data. Although, I also have the seemingly rare opinion here that ECC ram doesn't really matter on a laptop or desktop.

rowanG077 5y ago
You think it's oke that restarting your PC leads to data loss or corruption? That's basically a product killer for me. I reboot my laptop everyday.
- marcan_42 5y ago
  You presumably don't reboot your laptop by connecting a USB-PD gadget that issues a hard reset. A normal OS reboot is fine, that will flush the cache. The most common situation where this would affect laptops, in my experience so far, would be a broken driver causing a kernel lockup (not a panic) which triggers a watchdog reboot. That situation wouldn't allow for an NVMe flush.
  - rowanG077 5y ago
    Depends what exactly is a hard reboot. I don't reboot my laptop by issueing USB-PD command. But I do by holding the power button.
    - Cthulhu_ 5y ago
      See, that's a forced shutdown, a last resort measure; it's using a sledgehammer to tap in a nail. You shouldn't do that as a habit, even if this particular optimization issue wasn't a thing. I mean I grew up diligently turning off my PC by parking the disk and using the various operating system level shutdown procedures. Nowadays I smack the off button, but that still just triggers the OS shutdown procedure. I don't turn my Mac off as a rule, its sleep mode actually works. ish.
    - Thomaschaaf 5y ago
      Care to explain why?
      - rowanG077 5y ago
        Systemd takes 2 minutes to shutdown and I never got any way to resolve that.
        vgel 5y ago
        Linux Magic SysRq + R S E I V B key chord will immediately shut down while still properly flushing disk cache and such. A bit annoying to enter, but a handy tool to have in your toolbox.
        tashbarg 5y ago
        That's not the right keys and not the right order to do that. You should not flush caches before you terminated as much processes as possible correctly. And you are rebooting at the end. REISUB for a somewhat safe EMERGENCY reboot and O instead of B at the end for shutdown.
        vgel 5y ago
        Oh hmm, you're right. I've always done it with the other order and never had problems. Forget where I learned it that way...
      - olliej 5y ago
        If they're like me: outside of a software update I only reboot when the machine is not responding, at which point hard reboot is faster and more robust. I recognize it's not ideal, but I also don't think it's reasonable for the system to ever get to a point where I should be wanting to restart to "fix" it - and I would think it is a serious bug if doing so ever corrupted the system or lost any "saved" data.
    - marcan_42 5y ago
      I just tested that. Holding down the power button invokes a (somewhat special) btn_rst kernel panic before it has a chance to invoke a true hardware reset, and kernel panics involve an NVMe driver hook which I'm pretty sure issues a flush. Should be safe. At least re: this issue; it's still a bad idea because it's only safe if all software is written following data integrity and flush rules to the letter, and most software isn't. You're eventually going to run into issues on any OS by doing that, because most software doesn't get this right unless it's a database. And you're still going to lose data that's in buffer cache, I'm pretty sure that won't get flushed.
  - signal11 5y ago
    For products like the Mac Mini, which don’t have a battery, does this mean that a loss of mains power will cause data loss? Because brownouts do happen occasionally…
    - marcan_42 5y ago
      Yes. I've tested yanking the power and can easily see 5 seconds of data loss for data that was fsync()ed (but not full synced). I'm not sure yet if corruption due to reordering is also possible, but it seems likely.
- vondro 5y ago
  Shouldn't Mac OS issue flush on restart, as it does on sleep?
- Bud 5y ago
  1) A normal restart doesn't have this issue, at all. 2) Why are you rebooting a laptop daily? My uptime on my MacBook Pro averages 30-60 days. There's zero reason to reboot any modern OS daily.
  - lobocinza 5y ago
    > There's zero reason to reboot any modern OS daily. - I use Arch, I like to avoid accumulating too much major updates between reboots. - For a time I was facing a bug that resulted in a black screen of death after resuming sleep.
mordae 5y ago
It's not only losing a couple seconds of data. Write ordering does not work, meaning journals don't. You get a possibility of silent data corruption.
- londons_explore 5y ago
  But apple could quite easily fix write ordering
  - DooMMasteR 5y ago
    NVME even allows to make queues write through, so e.g. the kernel/fs driver could have/access the drive via a safe queue that always gets written. You can also prioritize queues to lower the chances of important data to be lost, though Apple seems to be super aggressive on caching and the drives tend to keep some written data in cache for quite long intervals.
koala_man 5y ago
> only allow ever losing 1 second of data For a database this means that every transaction will take a minimum of 1 second, otherwise you can't guarantee durability.

rowanG077 5y ago

You think it's oke that restarting your PC leads to data loss or corruption? That's basically a product killer for me. I reboot my laptop everyday.

marcan_42 5y ago
You presumably don't reboot your laptop by connecting a USB-PD gadget that issues a hard reset. A normal OS reboot is fine, that will flush the cache. The most common situation where this would affect laptops, in my experience so far, would be a broken driver causing a kernel lockup (not a panic) which triggers a watchdog reboot. That situation wouldn't allow for an NVMe flush.
- rowanG077 5y ago
  Depends what exactly is a hard reboot. I don't reboot my laptop by issueing USB-PD command. But I do by holding the power button.
  - Cthulhu_ 5y ago
    See, that's a forced shutdown, a last resort measure; it's using a sledgehammer to tap in a nail. You shouldn't do that as a habit, even if this particular optimization issue wasn't a thing. I mean I grew up diligently turning off my PC by parking the disk and using the various operating system level shutdown procedures. Nowadays I smack the off button, but that still just triggers the OS shutdown procedure. I don't turn my Mac off as a rule, its sleep mode actually works. ish.
  - Thomaschaaf 5y ago
    Care to explain why?
    - rowanG077 5y ago
      Systemd takes 2 minutes to shutdown and I never got any way to resolve that.
      - vgel 5y ago
        Linux Magic SysRq + R S E I V B key chord will immediately shut down while still properly flushing disk cache and such. A bit annoying to enter, but a handy tool to have in your toolbox.
        tashbarg 5y ago
        That's not the right keys and not the right order to do that. You should not flush caches before you terminated as much processes as possible correctly. And you are rebooting at the end. REISUB for a somewhat safe EMERGENCY reboot and O instead of B at the end for shutdown.
        vgel 5y ago
        Oh hmm, you're right. I've always done it with the other order and never had problems. Forget where I learned it that way...
    - olliej 5y ago
      If they're like me: outside of a software update I only reboot when the machine is not responding, at which point hard reboot is faster and more robust. I recognize it's not ideal, but I also don't think it's reasonable for the system to ever get to a point where I should be wanting to restart to "fix" it - and I would think it is a serious bug if doing so ever corrupted the system or lost any "saved" data.
  - marcan_42 5y ago
    I just tested that. Holding down the power button invokes a (somewhat special) btn_rst kernel panic before it has a chance to invoke a true hardware reset, and kernel panics involve an NVMe driver hook which I'm pretty sure issues a flush. Should be safe. At least re: this issue; it's still a bad idea because it's only safe if all software is written following data integrity and flush rules to the letter, and most software isn't. You're eventually going to run into issues on any OS by doing that, because most software doesn't get this right unless it's a database. And you're still going to lose data that's in buffer cache, I'm pretty sure that won't get flushed.
- signal11 5y ago
  For products like the Mac Mini, which don’t have a battery, does this mean that a loss of mains power will cause data loss? Because brownouts do happen occasionally…
  - marcan_42 5y ago
    Yes. I've tested yanking the power and can easily see 5 seconds of data loss for data that was fsync()ed (but not full synced). I'm not sure yet if corruption due to reordering is also possible, but it seems likely.
vondro 5y ago
Shouldn't Mac OS issue flush on restart, as it does on sleep?
Bud 5y ago
1) A normal restart doesn't have this issue, at all. 2) Why are you rebooting a laptop daily? My uptime on my MacBook Pro averages 30-60 days. There's zero reason to reboot any modern OS daily.
- lobocinza 5y ago
  > There's zero reason to reboot any modern OS daily. - I use Arch, I like to avoid accumulating too much major updates between reboots. - For a time I was facing a bug that resulted in a black screen of death after resuming sleep.

mordae 5y ago

It's not only losing a couple seconds of data. Write ordering does not work, meaning journals don't. You get a possibility of silent data corruption.

londons_explore 5y ago
But apple could quite easily fix write ordering
- DooMMasteR 5y ago
  NVME even allows to make queues write through, so e.g. the kernel/fs driver could have/access the drive via a safe queue that always gets written. You can also prioritize queues to lower the chances of important data to be lost, though Apple seems to be super aggressive on caching and the drives tend to keep some written data in cache for quite long intervals.

koala_man 5y ago

> only allow ever losing 1 second of data For a database this means that every transaction will take a minimum of 1 second, otherwise you can't guarantee durability.