3 ms·

Power-loss capacitors are needed because without them, flush commands issued by the OS return bogus values. Write confirmations are meaningless if the data isn'

Power-loss capacitors are needed because without them, flush commands issued by the OS return bogus values. Write confirmations are meaningless if the data isn't actually written to disk.

stefan_ 9y ago
Doesn't software like databases that care about this already assume the OS flush return values are bogus? And it still doesn't solve the problem of the host buffers disappearing. What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD?
- makmanalp 9y ago
  > Doesn't software like databases that care about this already assume the OS flush return values are bogus? Unsure of this (I would assume "it depends" and "probably not" because how else could it possibly know that anything is written to durable storage ever ever) but the idea is that if it didn't have to assume that life would be a lot easier. > What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD? If I understood your question correctly, databases and similar stuff write data in a way to try to solve exactly this problem, with write-ahead logs and such where you first write quickly and inexpensively to the log and then write to the database.
- imtringued 9y ago
  >Doesn't software like databases that care about this already assume the OS flush return values are bogus? The best a database can do is prevent corruption. It cannot magically prevent data loss of writes that have never reached the platter or NAND. >And it still doesn't solve the problem of the host buffers disappearing. What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD? You solve the problem of host buffers disappearing by using fsync. The scenario is that fsync works correctly on powerloss. Imagine you build an app consisting of 3 microservices. They are called Human, CPU and Disk. The human sends a request via a website to the CPU. The CPU wants to save the host buffer in a database and sends a fsync request to the Disk. The Disk sends an "OK" response immediately but actually saves the data in the background. The CPU sends an "OK" response to the human. Everything is fine. Suddenly powerloss happens and the disk cannot save the data in time. The database is smart, it knows how to reconstruct some of the data until the point where the transaction log has been cut off. Beyond that though the data is gone and the "OK" sent to the Human is no longer valid. Now imagine the same scenario with correct fsync. The CPU waits for the Disk and during a powerloss not only is the disk dead but the CPU is dead too and it therefore cannot send an "OK" response to the Human. What if you don't use fsync? What if you pipe your data to /dev/null? Well, here is the tough news: Nobody can help you. If you don't write data to disk then it's gone.
mindslight 9y ago
It's actually even worse than this, in that long-ago written data could be in the process of being refreshed/remapped when power is lost. The drive's software should be designed to work atomically to prevent this, but that's a whole lot of edge testing we're hoping a company will do on its closed software. Which is why things like cheap SD cards are notoriously unreliable, whereas Samsung SSDs are seen as decently reliable. What we really want is to get rid of these controller layers and let the OS directly manage the flash. But top of the line manufacturers are incentivized to decommodify their products to increase margins, and price-competitive manufacturers are inclined to stick to the same interface.
- deleted 9y ago
  [deleted]

Power-loss capacitors are needed because without them, flush commands issued by the OS return bogus values. Write confirmations are meaningless if the data isn't actually written to disk.

stefan_ 9y ago

Doesn't software like databases that care about this already assume the OS flush return values are bogus? And it still doesn't solve the problem of the host buffers disappearing. What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD?

makmanalp 9y ago
> Doesn't software like databases that care about this already assume the OS flush return values are bogus? Unsure of this (I would assume "it depends" and "probably not" because how else could it possibly know that anything is written to durable storage ever ever) but the idea is that if it didn't have to assume that life would be a lot easier. > What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD? If I understood your question correctly, databases and similar stuff write data in a way to try to solve exactly this problem, with write-ahead logs and such where you first write quickly and inexpensively to the log and then write to the database.
imtringued 9y ago
>Doesn't software like databases that care about this already assume the OS flush return values are bogus? The best a database can do is prevent corruption. It cannot magically prevent data loss of writes that have never reached the platter or NAND. >And it still doesn't solve the problem of the host buffers disappearing. What's the scenario, the host experiences a power-loss, but was somehow anticipating it and managed to stop writing to the SSD? You solve the problem of host buffers disappearing by using fsync. The scenario is that fsync works correctly on powerloss. Imagine you build an app consisting of 3 microservices. They are called Human, CPU and Disk. The human sends a request via a website to the CPU. The CPU wants to save the host buffer in a database and sends a fsync request to the Disk. The Disk sends an "OK" response immediately but actually saves the data in the background. The CPU sends an "OK" response to the human. Everything is fine. Suddenly powerloss happens and the disk cannot save the data in time. The database is smart, it knows how to reconstruct some of the data until the point where the transaction log has been cut off. Beyond that though the data is gone and the "OK" sent to the Human is no longer valid. Now imagine the same scenario with correct fsync. The CPU waits for the Disk and during a powerloss not only is the disk dead but the CPU is dead too and it therefore cannot send an "OK" response to the Human. What if you don't use fsync? What if you pipe your data to /dev/null? Well, here is the tough news: Nobody can help you. If you don't write data to disk then it's gone.

mindslight 9y ago

It's actually even worse than this, in that long-ago written data could be in the process of being refreshed/remapped when power is lost. The drive's software should be designed to work atomically to prevent this, but that's a whole lot of edge testing we're hoping a company will do on its closed software. Which is why things like cheap SD cards are notoriously unreliable, whereas Samsung SSDs are seen as decently reliable. What we really want is to get rid of these controller layers and let the OS directly manage the flash. But top of the line manufacturers are incentivized to decommodify their products to increase margins, and price-competitive manufacturers are inclined to stick to the same interface.

deleted 9y ago
[deleted]