5 ms·

Most electrical components have traditionally had open datasheets. It is only a relatively recent phenomenon that mass market ICs are locked behind NDAs.

sitzkrieg 3y ago
i only hit NDA on secure element chips and similar things. 99% of my embedding work is plain ol mcus w reams of datasheets available. i do not do pc stuff tho
- slaymaker1907 3y ago
  One common component where things are really secretive is with flash storage. This is because the underlying physical components are pretty commodified, and the software/firmware (like block virtual addressing) is where brands actually distinguish themselves. It's kind of unfortunate since there's a lot of really cool stuff you could do with more control over the hardware like reducing the size of a failing disk to extend its life instead of complex wear-leveling techniques.
  - jrockway 3y ago
    If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.
    - CrazyStat 3y ago
      > 1) Return to office. In the spirit of unbridled conspiracy theory (i.e. I have no evidence for this and don't believe it is generally true): Landlords are paying CxOs kickbacks to push return to office in order to prop up commercial real estate.
      - jrockway 3y ago
        Maybe. The most compelling reason I have heard is it's an easy way to get attrition. You don't have to do layoffs, which are expensive emotionally and financially.
      - srcreigh 3y ago
        In my city office buildings are being converted to residential housing. 3 buildings recently in an area of population size 250k.
        hoosieree 3y ago
        What part of the world? I've heard that the cost of converting office spaces to residential is more than building new.
        FridgeSeal 3y ago
        People keep telling me this too, so I’d be super interested to know if it wasn’t the case, because it really feels like a weak/overinflated reason.
        ethbr0 3y ago
        The reasons I've seen cited, chiefly individual unit plumbing, don't jive with modern building construction. Most office buildings are reinforced concrete floors, supported by internal load bearing columns, with glass facings. ... that's exactly the same as a hotel. Hotels have per-unit bathrooms, typically stacked on top of each other (floor over floor). Ergo, you can frame out/lose some space to cut several hotel-like vertical utility paths, plumb pipes from them to each floor's bathroom (or HVAC), and Bob's your uncle. Actual reasons would be things like office floors are built to lower load bearing standards than residential... which I don't believe is true.
        hoosieree 3y ago
        I found some papers [1,2] which claim conversions are not prohibitively expensive, though I assume not all office buildings are equally suitable. Open-plan office in particular seem impractical to convert. I wonder how many of the companies that built them that way to save money are now stuck with an empty building no one wants to buy? [1] https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb481336835e54f4fce3.pdf https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb4813... [2] https://journals.sagepub.com/doi/pdf/10.1080/00420980500380345 https://journals.sagepub.com/doi/pdf/10.1080/004209805003803...
    - peoplearepeople 3y ago
      (1) is really simple, managers want to lower the effort for themselves to interrupting people with sudden meetings
    - joezydeco 3y ago
      I think NAND controllers are fine. If erase is slow, especially at lower voltage levels, then buffer away. I also think internal controllers help mask and work around shitty yields on the arrays.
    - AnthonyMouse 3y ago
      > Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. The thing I don't get is, the chips are all commodities, and it's not like soldering them to a board is rocket science. Why isn't one of the companies that makes fully-specified inexpensive RISC-V chips selling one attached to some commodity flash chips and an NVMe connector? Include some minimalist open source firmware that offloads most of the work to the host OS and let the Linux community figure out how to make it better. At minimum it should allow you to undercut the existing competition on price because you're using a less expensive controller and no DRAM, and in a few years the open source drivers could have enough advantages over black box devices that nobody wants to buy anything else.
      - dezgeg 3y ago
        For starters, I doubt that acting as a NVMe device is something you can do with a off the self "inexpensive RISC-V" with any kind of acceptable performance. A NVME engine would almost certainly be something that the flash controller would have implemented in fixed function hardware. Also NAND flash practically requires some sort of error correction system - another thing that fixed function logic in a custom ASIC is very well suited for. Using CPU time on the host CPU for that would probably suck.
        AnthonyMouse 3y ago
        The basic function of transferring data between the flash chips and the PCIe bus shouldn't be a bottleneck. Dedicated silicon for error correction might be more efficient, but when there are already fast idle CPU cores it may not matter, and ECC algorithms can already be accelerated by SIMD instructions. Or for high performance machines add the dedicated silicon to the host CPU.
        dezgeg 3y ago
        These kind of high speed interfaces aren't just something that one just bitbangs on some microcontroller. I think even the first step, ie. finding a RISC-V chip that has capability of acting as a PCIe device at all will be a challenge. "Inexpensive" for sure not.
  - userbinator 3y ago
    IMHO the secrecy is more because they don't want people realising how incredibly unreliable NAND flash is becoming. 20 years of retention after 1M cycles? Advertise that proudly to everyone. 10 years after 100K cycles? Not too bad. 5 years after 10K cycles? OK. 3 months after 1K cycles? No, don't tell anyone. The very few leaked TLC/QLC flash datasheets out there don't even have any clear endurance numbers anymore.
  - sitzkrieg 3y ago
    i have done lots of flash storage and never hit an nda thankfully
- SV_BubbleTime 3y ago
  That is my experience, for NDA, Atmel wouldn’t let you see a datasheet without it. But, I’m starting to see fairly typical data sheets for micros being hidden away behind portals.
jrockway 3y ago
It really depends on the complexity. Sure, 555 timers and even microcontrollers have datasheets. CPUs that run Linux, Wifi chips supporting recent standards, etc.? Rare. This is why Linux in the 90s was a big pain; you get some random Ethernet chip and there is no documentation on how to talk to it, so you're just dead in the water. (I think Android was the turning point. From then on, stuff had to support Linux, because Mobile was the future and Apple wasn't going to buy your chip. But of course, "works on Linux" means that it works in the hardware vendor's 3-year-old version of Linux. And "works" is always up in the air when hardware vendors start writing code.) When I worked on embedded devices, we would report bugs and the vendor would send us new datasheets with updated known issues; updated to include the issue we reported. Or we'd do something like "the datasheet says it can handle a 100MHz clock but it's very unreliable" and they'd send back a datasheet saying it only supports 50MHz. That's the reason that datasheets are closed, every customer gets their own datasheet. The more you get into higher margin stuff, be it hardware or software, the more you'll run into datasheets or branches just for you. It is weird and inefficient. But cheaper than testing it yourself.
- reaperman 3y ago
  > But cheaper than testing it yourself. It always felt like, IMO, that we end up testing it ourselves anyways. Obviously we don't go through the whole spec, but like you, we implement what we need and find the relevant bugs/errata ourselves. It's very painful to run across these and I don't recall an MCU that doesn't have at least one that we run into and have to workaround because development is already deep enough that we're economically locked it and it wouldn't be worth the rework to switch to a different MCU. Sometimes the bugs are in the silicon, sometimes they're in the middleware, but both are very painful to root-cause.
  - themaninthedark 3y ago
    I think that they were saying it was cheaper for the manufacturer to not test it and use their customers as bug testers and troubleshooters.
- mfuzzey 3y ago

sitzkrieg 3y ago

i only hit NDA on secure element chips and similar things. 99% of my embedding work is plain ol mcus w reams of datasheets available. i do not do pc stuff tho

slaymaker1907 3y ago
One common component where things are really secretive is with flash storage. This is because the underlying physical components are pretty commodified, and the software/firmware (like block virtual addressing) is where brands actually distinguish themselves. It's kind of unfortunate since there's a lot of really cool stuff you could do with more control over the hardware like reducing the size of a failing disk to extend its life instead of complex wear-leveling techniques.
- jrockway 3y ago
  If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.
  - CrazyStat 3y ago
    > 1) Return to office. In the spirit of unbridled conspiracy theory (i.e. I have no evidence for this and don't believe it is generally true): Landlords are paying CxOs kickbacks to push return to office in order to prop up commercial real estate.
    - jrockway 3y ago
      Maybe. The most compelling reason I have heard is it's an easy way to get attrition. You don't have to do layoffs, which are expensive emotionally and financially.
    - srcreigh 3y ago
      In my city office buildings are being converted to residential housing. 3 buildings recently in an area of population size 250k.
      - hoosieree 3y ago
        What part of the world? I've heard that the cost of converting office spaces to residential is more than building new.
        FridgeSeal 3y ago
        People keep telling me this too, so I’d be super interested to know if it wasn’t the case, because it really feels like a weak/overinflated reason.
        ethbr0 3y ago
        The reasons I've seen cited, chiefly individual unit plumbing, don't jive with modern building construction. Most office buildings are reinforced concrete floors, supported by internal load bearing columns, with glass facings. ... that's exactly the same as a hotel. Hotels have per-unit bathrooms, typically stacked on top of each other (floor over floor). Ergo, you can frame out/lose some space to cut several hotel-like vertical utility paths, plumb pipes from them to each floor's bathroom (or HVAC), and Bob's your uncle. Actual reasons would be things like office floors are built to lower load bearing standards than residential... which I don't believe is true.
        hoosieree 3y ago
        I found some papers [1,2] which claim conversions are not prohibitively expensive, though I assume not all office buildings are equally suitable. Open-plan office in particular seem impractical to convert. I wonder how many of the companies that built them that way to save money are now stuck with an empty building no one wants to buy? [1] https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb481336835e54f4fce3.pdf https://pdfs.semanticscholar.org/da2d/49a0090e8631ff08eb4813... [2] https://journals.sagepub.com/doi/pdf/10.1080/00420980500380345 https://journals.sagepub.com/doi/pdf/10.1080/004209805003803...
  - peoplearepeople 3y ago
    (1) is really simple, managers want to lower the effort for themselves to interrupting people with sudden meetings
  - joezydeco 3y ago
    I think NAND controllers are fine. If erase is slow, especially at lower voltage levels, then buffer away. I also think internal controllers help mask and work around shitty yields on the arrays.
  - AnthonyMouse 3y ago
    > Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. The thing I don't get is, the chips are all commodities, and it's not like soldering them to a board is rocket science. Why isn't one of the companies that makes fully-specified inexpensive RISC-V chips selling one attached to some commodity flash chips and an NVMe connector? Include some minimalist open source firmware that offloads most of the work to the host OS and let the Linux community figure out how to make it better. At minimum it should allow you to undercut the existing competition on price because you're using a less expensive controller and no DRAM, and in a few years the open source drivers could have enough advantages over black box devices that nobody wants to buy anything else.
    - dezgeg 3y ago
      For starters, I doubt that acting as a NVMe device is something you can do with a off the self "inexpensive RISC-V" with any kind of acceptable performance. A NVME engine would almost certainly be something that the flash controller would have implemented in fixed function hardware. Also NAND flash practically requires some sort of error correction system - another thing that fixed function logic in a custom ASIC is very well suited for. Using CPU time on the host CPU for that would probably suck.
      - AnthonyMouse 3y ago
        The basic function of transferring data between the flash chips and the PCIe bus shouldn't be a bottleneck. Dedicated silicon for error correction might be more efficient, but when there are already fast idle CPU cores it may not matter, and ECC algorithms can already be accelerated by SIMD instructions. Or for high performance machines add the dedicated silicon to the host CPU.
        dezgeg 3y ago
        These kind of high speed interfaces aren't just something that one just bitbangs on some microcontroller. I think even the first step, ie. finding a RISC-V chip that has capability of acting as a PCIe device at all will be a challenge. "Inexpensive" for sure not.
- userbinator 3y ago
  IMHO the secrecy is more because they don't want people realising how incredibly unreliable NAND flash is becoming. 20 years of retention after 1M cycles? Advertise that proudly to everyone. 10 years after 100K cycles? Not too bad. 5 years after 10K cycles? OK. 3 months after 1K cycles? No, don't tell anyone. The very few leaked TLC/QLC flash datasheets out there don't even have any clear endurance numbers anymore.
- sitzkrieg 3y ago
  i have done lots of flash storage and never hit an nda thankfully
SV_BubbleTime 3y ago
That is my experience, for NDA, Atmel wouldn’t let you see a datasheet without it. But, I’m starting to see fairly typical data sheets for micros being hidden away behind portals.

jrockway 3y ago

It really depends on the complexity. Sure, 555 timers and even microcontrollers have datasheets. CPUs that run Linux, Wifi chips supporting recent standards, etc.? Rare. This is why Linux in the 90s was a big pain; you get some random Ethernet chip and there is no documentation on how to talk to it, so you're just dead in the water. (I think Android was the turning point. From then on, stuff had to support Linux, because Mobile was the future and Apple wasn't going to buy your chip. But of course, "works on Linux" means that it works in the hardware vendor's 3-year-old version of Linux. And "works" is always up in the air when hardware vendors start writing code.) When I worked on embedded devices, we would report bugs and the vendor would send us new datasheets with updated known issues; updated to include the issue we reported. Or we'd do something like "the datasheet says it can handle a 100MHz clock but it's very unreliable" and they'd send back a datasheet saying it only supports 50MHz. That's the reason that datasheets are closed, every customer gets their own datasheet. The more you get into higher margin stuff, be it hardware or software, the more you'll run into datasheets or branches just for you. It is weird and inefficient. But cheaper than testing it yourself.

reaperman 3y ago
> But cheaper than testing it yourself. It always felt like, IMO, that we end up testing it ourselves anyways. Obviously we don't go through the whole spec, but like you, we implement what we need and find the relevant bugs/errata ourselves. It's very painful to run across these and I don't recall an MCU that doesn't have at least one that we run into and have to workaround because development is already deep enough that we're economically locked it and it wouldn't be worth the rework to switch to a different MCU. Sometimes the bugs are in the silicon, sometimes they're in the middleware, but both are very painful to root-cause.
- themaninthedark 3y ago
  I think that they were saying it was cheaper for the manufacturer to not test it and use their customers as bug testers and troubleshooters.
mfuzzey 3y ago