5 ms·

This hack seems to take a 480GB SSD and transform it into a 120GB SSD However the write endurance (the amount of data you can write to the SSD before expecting

by nickcw 2y ago

This hack seems to take a 480GB SSD and transform it into a 120GB SSD However the write endurance (the amount of data you can write to the SSD before expecting failures) increases from 120TB to 4000TB which could be a very useful tradeoff, for example if you were using the disk to store logs. I've never seen this offered by the manufacturers though (maybe I haven't looked on the right place), I wonder why not?

dist-epoch 2y ago
Manufacturers offer that, in the form of TLC drives. Which are supported, unlike this hack which might cause data loss. This gives you 120GB with 4000TB write endurance, but you can buy a 4TB TLC drive with 3000TB write endurance for $200.
- greggsy 2y ago
  Then you could use this technique to achieve something like a 1.2TB disk with 40PB TBW? I’d be fascinated to hear any potential use cases for that level of endurance in modern data storage.
  - justinclift 2y ago
    > use cases for that level of endurance in modern data storage. All flash arrays. Saying that, as I have a bunch of smaller (400GB) 2.5" SAS SSDs combined into larger all-flash arrays, with each one of those SSD's rated for about 30PB of endurance. I'm expecting the servers to be dead by the time that endurance is exhausted though. ;)
    - greggsy 2y ago
      Exactly, I’ve done similar maths on my disks, and realised that it would be 20 years before they approach their end of life. By which point, they will be replaced for some new tech that will be cheaper, faster and more reliable and power efficient.
- causality0 2y ago
  Which drive would that be? The ones I'm seeing cost a lot more than $200.
  - hippich 2y ago
    I'll second that question!
  - BertoldVdb 2y ago
    SSD prices fluctuate a lot. I recently bought 4TB SSDs for 209eu but they are more expensive now (SNV2S/4000G, QLC though)
  - 5e92cb50239222b 2y ago
    My friend picked up a 3.84 TB Kingston SEDC600M with 7 PB of write endurance on sale for $180 a couple of months ago. That same place now sells them for around $360. Definitely an original drive. Maybe you just have to be on the lookout for one for when they go on sale.
greggsy 2y ago
I wonder if it would be useful as cache disks for ZFS or Synology (with further tinkering)?
- dgacmu 2y ago
  To dive slightly into that: You don't necessarily want to sacrifice space for a read cache disk: having more space can reduce writes as you do less replacement. But where you want endurance is for a ZIL SLOG (the write cache, effectively). Optane was great for this because of really high endurance and very low latency persistent writes, but, ... Farewell, dear optane, we barely knew you. The 400GB optane card had an endurance of 73 PB written. Pretty impressive, though at almost $3/GB it was really expensive. This would likely work but as a sibling commenter noted, you're probably better off with a purpose-built, high endurance drive. Since it's a write cache, just replace it a little early.
  - sneak 2y ago
    AIUI the slog is only for synchronous writes; most people using zfs at home don’t do any of those (unless you set sync=always which is not the default). https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/ https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/
- gdevenyi 2y ago
  Under provisioning is the standard recommendation for ZFS SSD cache/log/l2arc drives since those special types were a thing.
- liuliu 2y ago
  Optane 905p goes for $500 a piece (1T) I believe.
  - nine_k 2y ago
    For how long? Terrific for a hobby project, build farm, or even a business in a prototype stage (buy 3-4 then). Hardly acceptable in a larger setting where continuity in 10 years is important. Of course, not the exact same part available in 10 years (which is not unheard of, though), but something compatible or at least comparable.
BertoldVdb 2y ago
There are companies selling SLC SSDs (often using TLC or QLC flash but not using that mode) for industrial applications, for example Swissbit.
- userbinator 2y ago
  But they cost far more than what SLC should be expected to cost (4x the price of QLC or 3x the price of TLC.) The clear answer to the parent's question is planned obsolescence.
  - kllrnohj 2y ago
    > The clear answer to the parent's question is planned obsolescence. The clear answer is lack of demand. People want capacity more than they want endurance they aren't going to use. Early SSD deaths are not yet a common issue, certainly not with MLC or TLC anyway. But capacity is an issue
BearOso 2y ago
I don't understand how the author goes from 3.8 WAF(Write Amplication Factor) to 2.0 WAF and gets a 30x increase in endurance. I'd expect about 2x from that. From what I can see, he seems to be taking the 120TBW that the OEM warranties on the drive for the initial result, but then using the NAND's P/E cycles spec for the final result, which seems suspicious. The only thing that I could be missing is the NAND going to pSLC mode somehow increases the P/E cycles drastically, like requiring massively lower voltage to program the cells. But I think that would be included in the WAF measure. What am I missing?
- wtallis 2y ago
  QLC memory cells need to store and read back the voltage much more precisely than SLC memory cells. You get far more P/E cycles out of SLC because answering "is this a zero or a one?" remains fairly easy long after the cells are too worn to reliably distinguish between sixteen different voltage levels.
  - 1oooqooq 2y ago
    the author is wrong. what you mention is only true for actual SLC chip+firmware. qlc drivers probably don't even have the hardware to use the entire cell as slc, and they adopt one of N methods to save time/writes/power by underutilizing the resolution of the cell. neither gives you all the benefits, all increases the downsides to improve one upside. and you can't choose.
    - wtallis 2y ago
      Respectfully: to the extent that I can understand what you're trying to say, you don't seem to know what you're talking about. Stop trying so hard to bash the whole industry for making tradeoffs you don't agree with, and put a little more effort into understanding how these things actually work.
      - 1oooqooq 2y ago
        we are all here reading a machined translated article from a pt_br warez forum on using the wrong firmware on a ssd controller to talk to the firmware on a "smart" nand flash. to mimic a semblance of control of your own device. but yeah, I'm the delusional one and the industry is very sane and carrying for the wishes of the consumer. carry on.
1oooqooq 2y ago
data longevity depends on implementation in the firmware, which you have zero visibility. most consumer drivers will lower longevity.

by nickcw 2y ago

dist-epoch 2y ago

Manufacturers offer that, in the form of TLC drives. Which are supported, unlike this hack which might cause data loss. This gives you 120GB with 4000TB write endurance, but you can buy a 4TB TLC drive with 3000TB write endurance for $200.

greggsy 2y ago
Then you could use this technique to achieve something like a 1.2TB disk with 40PB TBW? I’d be fascinated to hear any potential use cases for that level of endurance in modern data storage.
- justinclift 2y ago
  > use cases for that level of endurance in modern data storage. All flash arrays. Saying that, as I have a bunch of smaller (400GB) 2.5" SAS SSDs combined into larger all-flash arrays, with each one of those SSD's rated for about 30PB of endurance. I'm expecting the servers to be dead by the time that endurance is exhausted though. ;)
  - greggsy 2y ago
    Exactly, I’ve done similar maths on my disks, and realised that it would be 20 years before they approach their end of life. By which point, they will be replaced for some new tech that will be cheaper, faster and more reliable and power efficient.
causality0 2y ago
Which drive would that be? The ones I'm seeing cost a lot more than $200.
- hippich 2y ago
  I'll second that question!
- BertoldVdb 2y ago
  SSD prices fluctuate a lot. I recently bought 4TB SSDs for 209eu but they are more expensive now (SNV2S/4000G, QLC though)
- 5e92cb50239222b 2y ago
  My friend picked up a 3.84 TB Kingston SEDC600M with 7 PB of write endurance on sale for $180 a couple of months ago. That same place now sells them for around $360. Definitely an original drive. Maybe you just have to be on the lookout for one for when they go on sale.

greggsy 2y ago

I wonder if it would be useful as cache disks for ZFS or Synology (with further tinkering)?

dgacmu 2y ago
To dive slightly into that: You don't necessarily want to sacrifice space for a read cache disk: having more space can reduce writes as you do less replacement. But where you want endurance is for a ZIL SLOG (the write cache, effectively). Optane was great for this because of really high endurance and very low latency persistent writes, but, ... Farewell, dear optane, we barely knew you. The 400GB optane card had an endurance of 73 PB written. Pretty impressive, though at almost $3/GB it was really expensive. This would likely work but as a sibling commenter noted, you're probably better off with a purpose-built, high endurance drive. Since it's a write cache, just replace it a little early.
- sneak 2y ago
  AIUI the slog is only for synchronous writes; most people using zfs at home don’t do any of those (unless you set sync=always which is not the default). https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/ https://jrs-s.net/2019/05/02/zfs-sync-async-zil-slog/
gdevenyi 2y ago
Under provisioning is the standard recommendation for ZFS SSD cache/log/l2arc drives since those special types were a thing.
liuliu 2y ago
Optane 905p goes for $500 a piece (1T) I believe.
- nine_k 2y ago
  For how long? Terrific for a hobby project, build farm, or even a business in a prototype stage (buy 3-4 then). Hardly acceptable in a larger setting where continuity in 10 years is important. Of course, not the exact same part available in 10 years (which is not unheard of, though), but something compatible or at least comparable.

BertoldVdb 2y ago

There are companies selling SLC SSDs (often using TLC or QLC flash but not using that mode) for industrial applications, for example Swissbit.

userbinator 2y ago
But they cost far more than what SLC should be expected to cost (4x the price of QLC or 3x the price of TLC.) The clear answer to the parent's question is planned obsolescence.
- kllrnohj 2y ago
  > The clear answer to the parent's question is planned obsolescence. The clear answer is lack of demand. People want capacity more than they want endurance they aren't going to use. Early SSD deaths are not yet a common issue, certainly not with MLC or TLC anyway. But capacity is an issue

BearOso 2y ago

I don't understand how the author goes from 3.8 WAF(Write Amplication Factor) to 2.0 WAF and gets a 30x increase in endurance. I'd expect about 2x from that. From what I can see, he seems to be taking the 120TBW that the OEM warranties on the drive for the initial result, but then using the NAND's P/E cycles spec for the final result, which seems suspicious. The only thing that I could be missing is the NAND going to pSLC mode somehow increases the P/E cycles drastically, like requiring massively lower voltage to program the cells. But I think that would be included in the WAF measure. What am I missing?

wtallis 2y ago
QLC memory cells need to store and read back the voltage much more precisely than SLC memory cells. You get far more P/E cycles out of SLC because answering "is this a zero or a one?" remains fairly easy long after the cells are too worn to reliably distinguish between sixteen different voltage levels.
- 1oooqooq 2y ago
  the author is wrong. what you mention is only true for actual SLC chip+firmware. qlc drivers probably don't even have the hardware to use the entire cell as slc, and they adopt one of N methods to save time/writes/power by underutilizing the resolution of the cell. neither gives you all the benefits, all increases the downsides to improve one upside. and you can't choose.
  - wtallis 2y ago
    Respectfully: to the extent that I can understand what you're trying to say, you don't seem to know what you're talking about. Stop trying so hard to bash the whole industry for making tradeoffs you don't agree with, and put a little more effort into understanding how these things actually work.
    - 1oooqooq 2y ago
      we are all here reading a machined translated article from a pt_br warez forum on using the wrong firmware on a ssd controller to talk to the firmware on a "smart" nand flash. to mimic a semblance of control of your own device. but yeah, I'm the delusional one and the industry is very sane and carrying for the wishes of the consumer. carry on.

1oooqooq 2y ago

data longevity depends on implementation in the firmware, which you have zero visibility. most consumer drivers will lower longevity.