10 ms·

One common component where things are really secretive is with flash storage. This is because the underlying physical components are pretty commodified, and the

by slaymaker1907 3y ago

One common component where things are really secretive is with flash storage. This is because the underlying physical components are pretty commodified, and the software/firmware (like block virtual addressing) is where brands actually distinguish themselves. It's kind of unfortunate since there's a lot of really cool stuff you could do with more control over the hardware like reducing the size of a failing disk to extend its life instead of complex wear-leveling techniques.

jrockway 3y ago
If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.
- CrazyStat 3y ago
  > 1) Return to office. In the spirit of unbridled conspiracy theory (i.e. I have no evidence for this and don't believe it is generally true): Landlords are paying CxOs kickbacks to push return to office in order to prop up commercial real estate.
  - jrockway 3y ago
    Maybe. The most compelling reason I have heard is it's an easy way to get attrition. You don't have to do layoffs, which are expensive emotionally and financially.
  - srcreigh 3y ago
    In my city office buildings are being converted to residential housing. 3 buildings recently in an area of population size 250k.
    - hoosieree 3y ago
      What part of the world? I've heard that the cost of converting office spaces to residential is more than building new.
      - FridgeSeal 3y ago
        People keep telling me this too, so I’d be super interested to know if it wasn’t the case, because it really feels like a weak/overinflated reason.
        ethbr0 3y ago
        The reasons I've seen cited, chiefly individual unit plumbing, don't jive with modern building construction. Most office buildings are reinforced concrete floors, supported by internal load bearing columns, with glass facings. ... that's exactly the same as a hotel. Hotels have per-unit bathrooms, typically stacked on top of each other (floor over floor). Ergo, you can frame out/lose some space to cut several hotel-like vertical utility paths, plumb pipes from them to each floor's bathroom (or HVAC), and Bob's your uncle. Actual reasons would be things like office floors are built to lower load bearing standards than residential... which I don't believe is true.
        hoosieree 3y ago
        I found some papers [1,2] which claim conversions are not prohibitively expensive, though I assume not all office buildings are equally suitable. Open-plan office in particular seem impractical to convert. I wonder how many of the companies that built them that way to save money are now stuck with an empty building no one wants to buy? [1] https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb481336835e54f4fce3.pdf https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb4813... [2] https://journals.sagepub.com/doi/pdf/10.1080/00420980500380345 https://journals.sagepub.com/doi/pdf/10.1080/004209805003803...
      - srcreigh 3y ago
        Waterloo Canada
  - Qwertious 3y ago
    I think that's basically true but it's not a conspiracy - take Apple, for instance. They have a $5 billion HQ on their balance sheet. If everyone works from home, then their investors will ask the CEO "why aren't you using this asset that costs $5 billion? It's inefficient - sell it off." The problem here is that if the CEO sells off the HQ (in a market where everyone has a bunch of unused real estate) they'll almost certainly take a loss (and potentially a loss of billions of dollars, if they e.g. can only sell it for $3B), whereas if they keep it then it's technically valued at what it originally cost to build. The CEO's job is to make the company (look) profitable, so the CEO wants to find any excuse to use the HQ, even if it loses a bit of money or doesn't otherwise make much sense. For companies that lease, it's the same problem or even worse - corporate office leases tend to be e.g. 3 years, and they're locked into paying actual dollars every quarter until the lease term ends, so if they can't claim to be using the office space then they're directly losing cash. Renegotiating the lease is a bad bet as the landlords have basically zero incentive to do so in the current environment. Thus, they want to appear to use their office space. There are some other niche benefits too - forcing return to office is a great way to reduce employee count without needing to claim you're downsizing (although if you want to downsize then you can totally make the claim), and it makes office perks more valuable. There are also some tax breaks from local governments that are contingent on having X number of workers in the office, and WFH could potentially require reversing the tax break. None of these are "conspiracies", they're just sparkling incentives.
- peoplearepeople 3y ago
  (1) is really simple, managers want to lower the effort for themselves to interrupting people with sudden meetings
- joezydeco 3y ago
  I think NAND controllers are fine. If erase is slow, especially at lower voltage levels, then buffer away. I also think internal controllers help mask and work around shitty yields on the arrays.
- AnthonyMouse 3y ago
  > Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. The thing I don't get is, the chips are all commodities, and it's not like soldering them to a board is rocket science. Why isn't one of the companies that makes fully-specified inexpensive RISC-V chips selling one attached to some commodity flash chips and an NVMe connector? Include some minimalist open source firmware that offloads most of the work to the host OS and let the Linux community figure out how to make it better. At minimum it should allow you to undercut the existing competition on price because you're using a less expensive controller and no DRAM, and in a few years the open source drivers could have enough advantages over black box devices that nobody wants to buy anything else.
  - dezgeg 3y ago
    For starters, I doubt that acting as a NVMe device is something you can do with a off the self "inexpensive RISC-V" with any kind of acceptable performance. A NVME engine would almost certainly be something that the flash controller would have implemented in fixed function hardware. Also NAND flash practically requires some sort of error correction system - another thing that fixed function logic in a custom ASIC is very well suited for. Using CPU time on the host CPU for that would probably suck.
    - AnthonyMouse 3y ago
      The basic function of transferring data between the flash chips and the PCIe bus shouldn't be a bottleneck. Dedicated silicon for error correction might be more efficient, but when there are already fast idle CPU cores it may not matter, and ECC algorithms can already be accelerated by SIMD instructions. Or for high performance machines add the dedicated silicon to the host CPU.
      - dezgeg 3y ago
        These kind of high speed interfaces aren't just something that one just bitbangs on some microcontroller. I think even the first step, ie. finding a RISC-V chip that has capability of acting as a PCIe device at all will be a challenge. "Inexpensive" for sure not.
userbinator 3y ago
IMHO the secrecy is more because they don't want people realising how incredibly unreliable NAND flash is becoming. 20 years of retention after 1M cycles? Advertise that proudly to everyone. 10 years after 100K cycles? Not too bad. 5 years after 10K cycles? OK. 3 months after 1K cycles? No, don't tell anyone. The very few leaked TLC/QLC flash datasheets out there don't even have any clear endurance numbers anymore.
sitzkrieg 3y ago
i have done lots of flash storage and never hit an nda thankfully

jrockway 3y ago

If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.

CrazyStat 3y ago
> 1) Return to office. In the spirit of unbridled conspiracy theory (i.e. I have no evidence for this and don't believe it is generally true): Landlords are paying CxOs kickbacks to push return to office in order to prop up commercial real estate.
- jrockway 3y ago
  Maybe. The most compelling reason I have heard is it's an easy way to get attrition. You don't have to do layoffs, which are expensive emotionally and financially.
- srcreigh 3y ago
  In my city office buildings are being converted to residential housing. 3 buildings recently in an area of population size 250k.
  - hoosieree 3y ago
    What part of the world? I've heard that the cost of converting office spaces to residential is more than building new.
    - FridgeSeal 3y ago
      People keep telling me this too, so I’d be super interested to know if it wasn’t the case, because it really feels like a weak/overinflated reason.
      - ethbr0 3y ago
        The reasons I've seen cited, chiefly individual unit plumbing, don't jive with modern building construction. Most office buildings are reinforced concrete floors, supported by internal load bearing columns, with glass facings. ... that's exactly the same as a hotel. Hotels have per-unit bathrooms, typically stacked on top of each other (floor over floor). Ergo, you can frame out/lose some space to cut several hotel-like vertical utility paths, plumb pipes from them to each floor's bathroom (or HVAC), and Bob's your uncle. Actual reasons would be things like office floors are built to lower load bearing standards than residential... which I don't believe is true.
        hoosieree 3y ago
        I found some papers [1,2] which claim conversions are not prohibitively expensive, though I assume not all office buildings are equally suitable. Open-plan office in particular seem impractical to convert. I wonder how many of the companies that built them that way to save money are now stuck with an empty building no one wants to buy? [1] https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb481336835e54f4fce3.pdf https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb4813... [2] https://journals.sagepub.com/doi/pdf/10.1080/00420980500380345 https://journals.sagepub.com/doi/pdf/10.1080/004209805003803...
    - srcreigh 3y ago
      Waterloo Canada
- Qwertious 3y ago
  I think that's basically true but it's not a conspiracy - take Apple, for instance. They have a $5 billion HQ on their balance sheet. If everyone works from home, then their investors will ask the CEO "why aren't you using this asset that costs $5 billion? It's inefficient - sell it off." The problem here is that if the CEO sells off the HQ (in a market where everyone has a bunch of unused real estate) they'll almost certainly take a loss (and potentially a loss of billions of dollars, if they e.g. can only sell it for $3B), whereas if they keep it then it's technically valued at what it originally cost to build. The CEO's job is to make the company (look) profitable, so the CEO wants to find any excuse to use the HQ, even if it loses a bit of money or doesn't otherwise make much sense. For companies that lease, it's the same problem or even worse - corporate office leases tend to be e.g. 3 years, and they're locked into paying actual dollars every quarter until the lease term ends, so if they can't claim to be using the office space then they're directly losing cash. Renegotiating the lease is a bad bet as the landlords have basically zero incentive to do so in the current environment. Thus, they want to appear to use their office space. There are some other niche benefits too - forcing return to office is a great way to reduce employee count without needing to claim you're downsizing (although if you want to downsize then you can totally make the claim), and it makes office perks more valuable. There are also some tax breaks from local governments that are contingent on having X number of workers in the office, and WFH could potentially require reversing the tax break. None of these are "conspiracies", they're just sparkling incentives.
peoplearepeople 3y ago
(1) is really simple, managers want to lower the effort for themselves to interrupting people with sudden meetings
joezydeco 3y ago
I think NAND controllers are fine. If erase is slow, especially at lower voltage levels, then buffer away. I also think internal controllers help mask and work around shitty yields on the arrays.
AnthonyMouse 3y ago
> Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. The thing I don't get is, the chips are all commodities, and it's not like soldering them to a board is rocket science. Why isn't one of the companies that makes fully-specified inexpensive RISC-V chips selling one attached to some commodity flash chips and an NVMe connector? Include some minimalist open source firmware that offloads most of the work to the host OS and let the Linux community figure out how to make it better. At minimum it should allow you to undercut the existing competition on price because you're using a less expensive controller and no DRAM, and in a few years the open source drivers could have enough advantages over black box devices that nobody wants to buy anything else.
- dezgeg 3y ago
  For starters, I doubt that acting as a NVMe device is something you can do with a off the self "inexpensive RISC-V" with any kind of acceptable performance. A NVME engine would almost certainly be something that the flash controller would have implemented in fixed function hardware. Also NAND flash practically requires some sort of error correction system - another thing that fixed function logic in a custom ASIC is very well suited for. Using CPU time on the host CPU for that would probably suck.
  - AnthonyMouse 3y ago
    The basic function of transferring data between the flash chips and the PCIe bus shouldn't be a bottleneck. Dedicated silicon for error correction might be more efficient, but when there are already fast idle CPU cores it may not matter, and ECC algorithms can already be accelerated by SIMD instructions. Or for high performance machines add the dedicated silicon to the host CPU.
    - dezgeg 3y ago
      These kind of high speed interfaces aren't just something that one just bitbangs on some microcontroller. I think even the first step, ie. finding a RISC-V chip that has capability of acting as a PCIe device at all will be a challenge. "Inexpensive" for sure not.

userbinator 3y ago

IMHO the secrecy is more because they don't want people realising how incredibly unreliable NAND flash is becoming. 20 years of retention after 1M cycles? Advertise that proudly to everyone. 10 years after 100K cycles? Not too bad. 5 years after 10K cycles? OK. 3 months after 1K cycles? No, don't tell anyone. The very few leaked TLC/QLC flash datasheets out there don't even have any clear endurance numbers anymore.

sitzkrieg 3y ago

i have done lots of flash storage and never hit an nda thankfully