13 ms·

Exploring ZFS ZIL with Intel Optane and NAND

121 points by olavgg 9y ago

unethical_ban 9y ago
I am certain the volume of acronyms is intentionally hilarious in the title, so let me pre-empt the complaining by laughing at it. Definitions are in the article.
- saagarjha 9y ago
  The article is self-deprecating in this aspect by squeezing in LOL as well: > yes we found a way to fit another acronym in the piece
- earenndil 9y ago
  ZFS = Z File System, a next-generation file system from oracle ZIL = ZFS Intent Log SSD = Solid State Drive, a type of long-term data storage system with better read/write speeds than an HDD NAND = not actually an acronym, but a type of logic gate. Not AND.
  - empthought 9y ago
    Hilariously enough, NAND is the only acronym. The others are initialisms.
    - ajnin 9y ago
      Initialisms are acronyms.
      - wnoise 9y ago
        Other way around. Acronyms are the subset of initialisms that are pronounceable.
        cmurf 9y ago
        Dictionary.com uses same definitions for both words, just in a different order. The first definition of acronym is when initials form a pronounced word; which is the second definition of initialism. So they're the same thing with the difference being vernacular.
        andoriyu 9y ago
        I'm 100% sure they both are subsets of abbreviations and the only difference is how they pronounced. I'm also pretty sure (not 100%) that due to misuse it's now okay to use them interchangeably according to some dictionaries.
    - NamTaf 9y ago
      You don't pronounce 'zill' and 'zef-ess' as acronyms?
  - brian-armstrong 9y ago
    Oracle? I think you mean Sun.
PascLeRasc 9y ago
I'm a little surprised that they didn't standardize the SSD capacity, or perform tests on empty vs nearly-full SSDs. Increasing size slightly decreases read speeds, just like L3 caches.
- benjaminl 9y ago
  The amount of space used by he ZIL on a SLOG is very modest. Typically less than a gigabyte. So a SSD used as a SLOG will be practically empty it's entire life. The reason one would use a large capacity SSD for this purpose is to take advantage of the large SSD's higher iops and greater wear leveling.
  - BrainInAJar 9y ago
    it's unfortunate but not surprising that you can't buy very small, very fast SSD's for this purpose
    - wmf 9y ago
      PMC-Sierra/Microsemi has some cool DRAM-based SSDs that do 1M IOPS (not a typo) with 16GB capacity. https://www.microsemi.com/product-directory/storage-boards/3690-flashtec-nvram-drives https://www.microsemi.com/product-directory/storage-boards/3... I don't think they're available in U.2 or M.2 format though. NVDIMM-N would probably also be great for slog. https://www.anandtech.com/show/12029/micron-announces-32gb-ddr4-nvdimmn-modules https://www.anandtech.com/show/12029/micron-announces-32gb-d...
      - carlob 9y ago
        Says over 10M IOPS in the link from microsemi, so… maybe a typo?
        wmf 9y ago
        1M in NVMe mode, 10M in nonstandard mode. I don't know if there's any software that can use the nonstandard mode.
tw04 9y ago
This article reeks of being "sponsored" by Intel, but there is no mention of it anywhere in the article. So my first question is - what, if any role did Intel play in the creation of this article? Second - I find it ironic after all the marketing buzz that Intel has finally admitted that the endurance of Optane is embarrassing compared to NAND devices. This was their original claim: >When Intel announced 3D XPoint, the company said that it would be 1,000 times faster than NAND flash, 10 times denser than (volatile) DRAM, and with 1,000 times the endurance of NAND Those claims are so far off from reality that I'm surprised their shareholders haven't sued them over the claims.
- jlgaddis 9y ago
  Perhaps you missed this part? > The genesis of this project was that a user requested we setup a custom demo in our DemoEval lab to compare drives using this specific workload. The individual wanted to compare a few of their existing NAND solutions to Optane.
- tpetry 9y ago
  Why should it be an advertisment bought by intel? Optane disks are technically the best at the moment for ZFS SLOG but nobody had tested it in reality yet, so they just did it very good.
- wmf 9y ago
  ServeTheHome is one of the most honest hardware review sites, especially in the very small field of server/enterprise hardware. AFAIK ServeTheHome is basically advertising for DemoEval which is an independent lab full of servers that you can rent access to. So there's a commercial interest behind it, but it's not Intel. Here's their policy if you want to believe it: https://www.servethehome.com/about/editorial-copyright-policies/ https://www.servethehome.com/about/editorial-copyright-polic... Looking at Optane specifically, it is just that good. It is 2x-10x better than NAND (at ~2x the price) in certain aspects like mixed read/write or fsync. This is very far from Intel's 1000x claims, but Optane is still worth buying in some cases. (Also, feel free to call me a paid shill if you want to get deeper in the mud.)
  - 9y ago
brian-armstrong 9y ago
So we know the CPUs have IME. What else? Is it conceivable that SSDs do as well? Is any Intel product safe to use?
- wmf 9y ago
  No, nothing is safe. There's an NSA backdoor in your mouse.
  - r00fus 9y ago
    you jest, certainly, yet... now I'm not sure.
    - arca_vorago 9y ago
      They are called implants.[1] To GP, yes even hdds, I suspect similar with ssds.[2] 1: https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html https://www.schneier.com/blog/archives/2014/03/cottonmouth-i... 2: http://spritesmods.com/?art=hddhack&page=1 http://spritesmods.com/?art=hddhack&page=1

121 points by olavgg 9y ago

I am certain the volume of acronyms is intentionally hilarious in the title, so let me pre-empt the complaining by laughing at it. Definitions are in the article.

saagarjha 9y ago
The article is self-deprecating in this aspect by squeezing in LOL as well: > yes we found a way to fit another acronym in the piece
earenndil 9y ago
ZFS = Z File System, a next-generation file system from oracle ZIL = ZFS Intent Log SSD = Solid State Drive, a type of long-term data storage system with better read/write speeds than an HDD NAND = not actually an acronym, but a type of logic gate. Not AND.
- empthought 9y ago
  Hilariously enough, NAND is the only acronym. The others are initialisms.
  - ajnin 9y ago
    Initialisms are acronyms.
    - wnoise 9y ago
      Other way around. Acronyms are the subset of initialisms that are pronounceable.
      - cmurf 9y ago
        Dictionary.com uses same definitions for both words, just in a different order. The first definition of acronym is when initials form a pronounced word; which is the second definition of initialism. So they're the same thing with the difference being vernacular.
      - andoriyu 9y ago
        I'm 100% sure they both are subsets of abbreviations and the only difference is how they pronounced. I'm also pretty sure (not 100%) that due to misuse it's now okay to use them interchangeably according to some dictionaries.
  - NamTaf 9y ago
    You don't pronounce 'zill' and 'zef-ess' as acronyms?
- brian-armstrong 9y ago
  Oracle? I think you mean Sun.

PascLeRasc 9y ago

I'm a little surprised that they didn't standardize the SSD capacity, or perform tests on empty vs nearly-full SSDs. Increasing size slightly decreases read speeds, just like L3 caches.

benjaminl 9y ago
The amount of space used by he ZIL on a SLOG is very modest. Typically less than a gigabyte. So a SSD used as a SLOG will be practically empty it's entire life. The reason one would use a large capacity SSD for this purpose is to take advantage of the large SSD's higher iops and greater wear leveling.
- BrainInAJar 9y ago
  it's unfortunate but not surprising that you can't buy very small, very fast SSD's for this purpose
  - wmf 9y ago
    PMC-Sierra/Microsemi has some cool DRAM-based SSDs that do 1M IOPS (not a typo) with 16GB capacity. https://www.microsemi.com/product-directory/storage-boards/3690-flashtec-nvram-drives https://www.microsemi.com/product-directory/storage-boards/3... I don't think they're available in U.2 or M.2 format though. NVDIMM-N would probably also be great for slog. https://www.anandtech.com/show/12029/micron-announces-32gb-ddr4-nvdimmn-modules https://www.anandtech.com/show/12029/micron-announces-32gb-d...
    - carlob 9y ago
      Says over 10M IOPS in the link from microsemi, so… maybe a typo?
      - wmf 9y ago
        1M in NVMe mode, 10M in nonstandard mode. I don't know if there's any software that can use the nonstandard mode.

tw04 9y ago

This article reeks of being "sponsored" by Intel, but there is no mention of it anywhere in the article. So my first question is - what, if any role did Intel play in the creation of this article? Second - I find it ironic after all the marketing buzz that Intel has finally admitted that the endurance of Optane is embarrassing compared to NAND devices. This was their original claim: >When Intel announced 3D XPoint, the company said that it would be 1,000 times faster than NAND flash, 10 times denser than (volatile) DRAM, and with 1,000 times the endurance of NAND Those claims are so far off from reality that I'm surprised their shareholders haven't sued them over the claims.

jlgaddis 9y ago
Perhaps you missed this part? > The genesis of this project was that a user requested we setup a custom demo in our DemoEval lab to compare drives using this specific workload. The individual wanted to compare a few of their existing NAND solutions to Optane.
tpetry 9y ago
Why should it be an advertisment bought by intel? Optane disks are technically the best at the moment for ZFS SLOG but nobody had tested it in reality yet, so they just did it very good.
wmf 9y ago
ServeTheHome is one of the most honest hardware review sites, especially in the very small field of server/enterprise hardware. AFAIK ServeTheHome is basically advertising for DemoEval which is an independent lab full of servers that you can rent access to. So there's a commercial interest behind it, but it's not Intel. Here's their policy if you want to believe it: https://www.servethehome.com/about/editorial-copyright-policies/ https://www.servethehome.com/about/editorial-copyright-polic... Looking at Optane specifically, it is just that good. It is 2x-10x better than NAND (at ~2x the price) in certain aspects like mixed read/write or fsync. This is very far from Intel's 1000x claims, but Optane is still worth buying in some cases. (Also, feel free to call me a paid shill if you want to get deeper in the mud.)
- 9y ago

brian-armstrong 9y ago

So we know the CPUs have IME. What else? Is it conceivable that SSDs do as well? Is any Intel product safe to use?

wmf 9y ago
No, nothing is safe. There's an NSA backdoor in your mouse.
- r00fus 9y ago
  you jest, certainly, yet... now I'm not sure.
  - arca_vorago 9y ago
    They are called implants.[1] To GP, yes even hdds, I suspect similar with ssds.[2] 1: https://www.schneier.com/blog/archives/2014/03/cottonmouth-i_n.html https://www.schneier.com/blog/archives/2014/03/cottonmouth-i... 2: http://spritesmods.com/?art=hddhack&page=1 http://spritesmods.com/?art=hddhack&page=1