6 ms·

I'm surprised we haven't seen a smaller than M.2 form factor for SATA on embedded devices. It seems like the big limiter on devices like the Raspberry Pi and ot

I'm surprised we haven't seen a smaller than M.2 form factor for SATA on embedded devices. It seems like the big limiter on devices like the Raspberry Pi and others are the sd cards/emmc modules, since the amount processing power and ram have been continually going up. If you need disk performance on the Pi 4, wouldn't it make more sense to only keep the kernel,initrd on the sdcard and mount the primary filesystem off a USB3 device (like an mSATA-2-USB enclosure), that is if you have the space to spare for it in your project/enclosure? I did something similar with one of the SolidRun devices, but I had to scuttle it due to other hardware problems: https://penguindreams.org/blog/review-clearfog-pro/ https://penguindreams.org/blog/review-clearfog-pro/

jacobush 7y ago
Yep. USB disks also are generally much more robust to errors than SD cards. My mail spool is on USB, the OS is on the SD card.
- geerlingguy 7y ago
  Soon (hopefully) the Pi 4 will be able to completely boot off USB 3.0 devices; right now you can partially boot off an external volume, but /boot still has to be on a microSD card. Though in my testing, it's still not a night-and-day performance difference, if judged by the Pi 3B+. I am not sure if the Pi 4's USB 3.0 port bandwidth will give that much of an improvement. It would be quite nice to have any form of drive supported more directly, NVMe, m.2, whatever.
  - NikolaeVarius 7y ago
    Does /boot get read/written to enough where this is not "good enough"
    - geerlingguy 7y ago
      It doesn't, but in my informal tests (I haven't had time to set up a thorough benchmark) real-world usage of an external USB 3.0 SSD (testing with a cheap Kingston SSD, though, and a cheap USB SATA interface) is only marginally faster than a Samsung Evo+ microSD card.
      - kup0 7y ago
        I wonder if this is also due to the fact that many desktop SSDs do not focus on 4k read/write performance. This is one big area where nvme SSDs seem to have an advantage, their 4k read/write is usually significantly higher than that of other SSDs (at least from benchmarks I've seen). Maybe even a lower end nvme ssd + usb enclosure would be the way to go to get the best performance. Unless there's something about the USB interface itself that kills random 4k r/w performance?
        geerlingguy 7y ago
        True; some of the cheapest SSDs I've tested seem to just be the same package that would be inside a USB flash drive or SD card, hardwired to an SATA plug in a giant 2.5" enclosure!
jdietrich 7y ago
The SD 7.0 standard includes support for PCIe and NVMe. https://www.sdcard.org/press/SD_EXPRESS_A_REVOLUTIONARY_INNOVATION_FOR_SD_MEMORY_CARDS.pdf https://www.sdcard.org/press/SD_EXPRESS_A_REVOLUTIONARY_INNO...
- geerlingguy 7y ago
  Good chance that doesn't trickle down to the microSD interface; it seems like the best performance improvements are reserved for the full-size SD cards, likely because they can fit more thermal mass and connection pins in the larger case.
  - jws 7y ago
    The term you want is “microSD express”. That is the standard for PCIe and NVME on microSD cards. It specs out at about 1GB/second transfer rates. Fun engineering decisions: It can use RAM from the hosting computer to store its logical block mapping tables so it doesn’t have to have DRAM onboard for that. That’s a clever way to use what the host has in abundance to avoid putting it on the price and volume constrained microSD card. Now… how you are going to get the heat out of it when it is running at 1GB/sec. I wonder if we will see new microSD sockets with integrated heat sinks.
    - jononor 7y ago
      Solder the microSD and use the solder-points for thermal connection to the board, like for most other medium-power ICs. It is also cheaper than a socket.
      - elcritch 7y ago
        Can standard microSD's survive standard reflow soldering? I wouldn't think they survive, but would be nice.
        jononor 7y ago
jononor 7y ago
General practice in embedded is to avoid writing to disk as much as possible. Data collection is mostly done by sending it to the cloud for storage there. However 'edge processing' and 'edge storage' are hot topics, so there will probably be more work in this area. Some of the surveillance camera vendors are showing interest in this, since it would be beneficial to reduce data needed to be sent. Especially since vast majority of the data is never looked at, or only looked at once (human/machine event detection). But one has the (fundamental?) problem that the sensor is often installed in an adversarial environment. Would be sad to try to pull data after a crime, only to realize the camera storage was destroyed.
- jononor 7y ago
  Many embedded applications are also not particularly form-factor constrained. A good security camera lens is often easily bigger than a couple of 2.5" hard-drives and a medium sized single-board computation. Heck, just wiring boxes for an industrial sensor system, are can often bigger than 10s of 3.5" disks.
  - sigstoat 7y ago
    > Many embedded applications are also not particularly form-factor constrained. just because the overall package is large doesn't mean the embedded designers are allocated a large portion of that. and the upstream suppliers are always going to be pushing to be a bit smaller, so they can fit into more things, and get more sales.
  - sebcat 7y ago
    Heat and power (think PoE class) are other constraints to take into consideration.
- oguzb 7y ago
  Agreed, but Raspbian is not an embedded operating system. It spams the syslog and uses a journalling filesystem.
- nitrogen 7y ago
  General practice in embedded is to avoid writing to disk as much as possible. Yes. Data collection is mostly done by sending it to the cloud for storage there. Nononono, what has the world come to? However 'edge processing' and 'edge storage' are hot topics Meaning "how it used to be done when hardware was a hundred times slower"?
zamadatix 7y ago
SATA DOM can get pretty tiny, https://www.neweggbusiness.com/Product/Product.aspx?Item=9SIV0M23N32299 https://www.neweggbusiness.com/Product/Product.aspx?Item=9SI... Raspberry Pi has supported direct USB boot on most boards for a while now (Pi 4 awaiting a boot firmware update), no need to also use a uSD. Glad to hear I'm not the only one who had a nightmare experience trying to use the ClearFog Pro as a router :).
nrp 7y ago
There's also the UFS Card format, which is a card version of the internal storage that most smartphones use these days. It's still nascent, but in theory should trickle down to single board computers as low cost SoCs get UFS interfaces. It's substantially higher perf than eMMC/SD at read, write, and IOPS.

jacobush 7y ago

Yep. USB disks also are generally much more robust to errors than SD cards. My mail spool is on USB, the OS is on the SD card.

geerlingguy 7y ago
Soon (hopefully) the Pi 4 will be able to completely boot off USB 3.0 devices; right now you can partially boot off an external volume, but /boot still has to be on a microSD card. Though in my testing, it's still not a night-and-day performance difference, if judged by the Pi 3B+. I am not sure if the Pi 4's USB 3.0 port bandwidth will give that much of an improvement. It would be quite nice to have any form of drive supported more directly, NVMe, m.2, whatever.
- NikolaeVarius 7y ago
  Does /boot get read/written to enough where this is not "good enough"
  - geerlingguy 7y ago
    It doesn't, but in my informal tests (I haven't had time to set up a thorough benchmark) real-world usage of an external USB 3.0 SSD (testing with a cheap Kingston SSD, though, and a cheap USB SATA interface) is only marginally faster than a Samsung Evo+ microSD card.
    - kup0 7y ago
      I wonder if this is also due to the fact that many desktop SSDs do not focus on 4k read/write performance. This is one big area where nvme SSDs seem to have an advantage, their 4k read/write is usually significantly higher than that of other SSDs (at least from benchmarks I've seen). Maybe even a lower end nvme ssd + usb enclosure would be the way to go to get the best performance. Unless there's something about the USB interface itself that kills random 4k r/w performance?
      - geerlingguy 7y ago
        True; some of the cheapest SSDs I've tested seem to just be the same package that would be inside a USB flash drive or SD card, hardwired to an SATA plug in a giant 2.5" enclosure!

jdietrich 7y ago

The SD 7.0 standard includes support for PCIe and NVMe. https://www.sdcard.org/press/SD_EXPRESS_A_REVOLUTIONARY_INNOVATION_FOR_SD_MEMORY_CARDS.pdf https://www.sdcard.org/press/SD_EXPRESS_A_REVOLUTIONARY_INNO...

geerlingguy 7y ago
Good chance that doesn't trickle down to the microSD interface; it seems like the best performance improvements are reserved for the full-size SD cards, likely because they can fit more thermal mass and connection pins in the larger case.
- jws 7y ago
  The term you want is “microSD express”. That is the standard for PCIe and NVME on microSD cards. It specs out at about 1GB/second transfer rates. Fun engineering decisions: It can use RAM from the hosting computer to store its logical block mapping tables so it doesn’t have to have DRAM onboard for that. That’s a clever way to use what the host has in abundance to avoid putting it on the price and volume constrained microSD card. Now… how you are going to get the heat out of it when it is running at 1GB/sec. I wonder if we will see new microSD sockets with integrated heat sinks.
  - jononor 7y ago
    Solder the microSD and use the solder-points for thermal connection to the board, like for most other medium-power ICs. It is also cheaper than a socket.
    - elcritch 7y ago
      Can standard microSD's survive standard reflow soldering? I wouldn't think they survive, but would be nice.
      - jononor 7y ago

jononor 7y ago

General practice in embedded is to avoid writing to disk as much as possible. Data collection is mostly done by sending it to the cloud for storage there. However 'edge processing' and 'edge storage' are hot topics, so there will probably be more work in this area. Some of the surveillance camera vendors are showing interest in this, since it would be beneficial to reduce data needed to be sent. Especially since vast majority of the data is never looked at, or only looked at once (human/machine event detection). But one has the (fundamental?) problem that the sensor is often installed in an adversarial environment. Would be sad to try to pull data after a crime, only to realize the camera storage was destroyed.

jononor 7y ago
Many embedded applications are also not particularly form-factor constrained. A good security camera lens is often easily bigger than a couple of 2.5" hard-drives and a medium sized single-board computation. Heck, just wiring boxes for an industrial sensor system, are can often bigger than 10s of 3.5" disks.
- sigstoat 7y ago
  > Many embedded applications are also not particularly form-factor constrained. just because the overall package is large doesn't mean the embedded designers are allocated a large portion of that. and the upstream suppliers are always going to be pushing to be a bit smaller, so they can fit into more things, and get more sales.
- sebcat 7y ago
  Heat and power (think PoE class) are other constraints to take into consideration.
oguzb 7y ago
Agreed, but Raspbian is not an embedded operating system. It spams the syslog and uses a journalling filesystem.
nitrogen 7y ago
General practice in embedded is to avoid writing to disk as much as possible. Yes. Data collection is mostly done by sending it to the cloud for storage there. Nononono, what has the world come to? However 'edge processing' and 'edge storage' are hot topics Meaning "how it used to be done when hardware was a hundred times slower"?

zamadatix 7y ago

SATA DOM can get pretty tiny, https://www.neweggbusiness.com/Product/Product.aspx?Item=9SIV0M23N32299 https://www.neweggbusiness.com/Product/Product.aspx?Item=9SI... Raspberry Pi has supported direct USB boot on most boards for a while now (Pi 4 awaiting a boot firmware update), no need to also use a uSD. Glad to hear I'm not the only one who had a nightmare experience trying to use the ClearFog Pro as a router :).

nrp 7y ago

There's also the UFS Card format, which is a card version of the internal storage that most smartphones use these days. It's still nascent, but in theory should trickle down to single board computers as low cost SoCs get UFS interfaces. It's substantially higher perf than eMMC/SD at read, write, and IOPS.