8 ms·

The NTSB's original report has more detail on how the SD Card was encrypted and how the NTSB managed to decrypt it: https://data.ntsb.gov/Docket/Document/docBL

The NTSB's original report has more detail on how the SD Card was encrypted and how the NTSB managed to decrypt it: https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&FileExtension=pdf&FileName=Underwater%20Camera%20-%20Specialist%27s%20Factual%20Report-Rel.pdf https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...

jeffrallen 1y ago
Does not leave SubC in a particularly flattering light...
- Aurornis 1y ago
  They had no idea how their own product worked. They didn’t even know it used encrypted storage. This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.
  - ryandrake 1y ago
    The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."
    - throwaway173738 1y ago
      That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
      - bschwindHN 1y ago
        What kind of hardware projects do you work on? I'm mostly in the software space but in the past few years I've been doing a lot more embedded stuff, and the trend I notice is that companies are making great hardware, and then completely ruining its usefulness with bad software and firmware. It's kind of mind blowing to me because I always considered software to be the easy part of making a product, compared to, you know, etching microscopic patterns onto sand to make magical transistors appear in just the right way to do the task you want.
        kiba 1y ago
        It's all about what works enough according to a rather low bar.
        homebrewer 1y ago
        It's almost as if hardware was developed by actual engineers, unlike, you know, the other half of the pie.
      - lexszero_ 1y ago
        > Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive. While I agree with your sentiment that there's a lot of poor software engineering in embedded space (especially in consumer-oriented novelty products, less so in established fields like industrial or telco), I can't but wonder what's wrong with Yocto? In my experience, it's quite the opposite: Yocto is the quickest path to get the firmware for a new device assembled, once you have climbed its pretty steep learning curve. I have built a few homebrew firmware build systems out of Debian and make/shell scripts (not my choice), you pretty quickly find yourself reinventing half of the stuff that Yocto does out of the box, but it's all bespoke, janky and hard to maintain. While with Yocto you just take the vendor's meta layer for BSP, put your application in another, and it bakes you a set of flashable images on the other end, complete with SDKs and other goodies for your dev workflow, reproducibly. It doesn't get significantly more sophisticated once you start to need kernel customizations, firmware updates with A/B partition layout, readonly rootfs, manage board- or customer-specific variants and other features that are very common in embedded systems but poorly or not at all supported in standard distros.
    - dboreham 1y ago
      This is in fact the case.
- Symbiote 1y ago
  It survived the pressure, does the rest matter?
  - nativeit 1y ago
    In this context? Certainly. This is more examples of their using untested, uncertified equipment. Anyone who’s ever worked in O2-enriched pressurized environments should at least be instantly concerned about whether this device had been properly rated for fire prevention.
mk_stjames 1y ago
The System on Module board is an Inforce 6601 SOM. [0] It uses a Qualcomm Snapdragon 820 and they provide prebuilt Ubuntu Linaro distros for it, preconfigured for the board. The camera manufacturer likely just tossed it straight in as configured and thus didn't know how the full disk encryption was setup. This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'. See also: almost any specialty medical electronic device ever manufactured. [0] https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-android-on-snapdragon-820/ https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-androi...
- Interesco 1y ago
  The 3D-printed (and hot glued?) part in Figure 3 further support this theory (not that 3D prints can't be used in production).
- userbinator 1y ago
  Indeed this is massively overcomplicated, as one only needs to see what dashcams use to know that you don't need, or perhaps want, an entire OS on it.
- Neywiny 1y ago
  I'll admit I only watched a video on it not the report, but it had pictures reportedly redacted at manufacturer request. It showed a teensy 3 and some adafruit qwiic board in there. Obviously the real engineering is in the enclosure. Otherwise it could just be a webcam. But still, it's clearly not a very in depth electrical design. I'm all for SoMs if you can but they don't guarantee you the adventure of custom hardware bringing moving through all the software stacks and whatnot.
  - 15155 1y ago
    No serious commercial product should be using a Teensy under basically any circumstance.

jeffrallen 1y ago

Does not leave SubC in a particularly flattering light...

Aurornis 1y ago
They had no idea how their own product worked. They didn’t even know it used encrypted storage. This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.
- ryandrake 1y ago
  The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."
  - throwaway173738 1y ago
    That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
    - bschwindHN 1y ago
      What kind of hardware projects do you work on? I'm mostly in the software space but in the past few years I've been doing a lot more embedded stuff, and the trend I notice is that companies are making great hardware, and then completely ruining its usefulness with bad software and firmware. It's kind of mind blowing to me because I always considered software to be the easy part of making a product, compared to, you know, etching microscopic patterns onto sand to make magical transistors appear in just the right way to do the task you want.
      - kiba 1y ago
        It's all about what works enough according to a rather low bar.
      - homebrewer 1y ago
        It's almost as if hardware was developed by actual engineers, unlike, you know, the other half of the pie.
    - lexszero_ 1y ago
      > Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive. While I agree with your sentiment that there's a lot of poor software engineering in embedded space (especially in consumer-oriented novelty products, less so in established fields like industrial or telco), I can't but wonder what's wrong with Yocto? In my experience, it's quite the opposite: Yocto is the quickest path to get the firmware for a new device assembled, once you have climbed its pretty steep learning curve. I have built a few homebrew firmware build systems out of Debian and make/shell scripts (not my choice), you pretty quickly find yourself reinventing half of the stuff that Yocto does out of the box, but it's all bespoke, janky and hard to maintain. While with Yocto you just take the vendor's meta layer for BSP, put your application in another, and it bakes you a set of flashable images on the other end, complete with SDKs and other goodies for your dev workflow, reproducibly. It doesn't get significantly more sophisticated once you start to need kernel customizations, firmware updates with A/B partition layout, readonly rootfs, manage board- or customer-specific variants and other features that are very common in embedded systems but poorly or not at all supported in standard distros.
  - dboreham 1y ago
    This is in fact the case.
Symbiote 1y ago
It survived the pressure, does the rest matter?
- nativeit 1y ago
  In this context? Certainly. This is more examples of their using untested, uncertified equipment. Anyone who’s ever worked in O2-enriched pressurized environments should at least be instantly concerned about whether this device had been properly rated for fire prevention.

mk_stjames 1y ago

The System on Module board is an Inforce 6601 SOM. [0] It uses a Qualcomm Snapdragon 820 and they provide prebuilt Ubuntu Linaro distros for it, preconfigured for the board. The camera manufacturer likely just tossed it straight in as configured and thus didn't know how the full disk encryption was setup. This whole camera design looks like one of those 'we gave this project to some undergrad engineering students who've never designed a commercial product before and had no price target and thus it has a whole damn embedded linux system inside it for merely taking some HD video and stills triggered by some external wiring and saving them to an SD card'. See also: almost any specialty medical electronic device ever manufactured. [0] https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-android-on-snapdragon-820/ https://linuxgizmos.com/tiny-rugged-com-runs-linux-or-androi...

Interesco 1y ago
The 3D-printed (and hot glued?) part in Figure 3 further support this theory (not that 3D prints can't be used in production).
userbinator 1y ago
Indeed this is massively overcomplicated, as one only needs to see what dashcams use to know that you don't need, or perhaps want, an entire OS on it.
Neywiny 1y ago
I'll admit I only watched a video on it not the report, but it had pictures reportedly redacted at manufacturer request. It showed a teensy 3 and some adafruit qwiic board in there. Obviously the real engineering is in the enclosure. Otherwise it could just be a webcam. But still, it's clearly not a very in depth electrical design. I'm all for SoMs if you can but they don't guarantee you the adventure of custom hardware bringing moving through all the software stacks and whatnot.
- 15155 1y ago
  No serious commercial product should be using a Teensy under basically any circumstance.