7 ms·

Totally fair question! My reasons were: 1. To maximize Photon's battery life I tried to make the hardware do the absolute minimum to record imagery. To that en

Totally fair question! My reasons were: 1. To maximize Photon's battery life I tried to make the hardware do the absolute minimum to record imagery. To that end, Photon stores images on an unformatted SD card (ie one huge linear array of bytes) which is easy and fast to index into. 2. A 16-bit MSP430 handles writing the image data, and the MSP430's codespace is 99% exhausted without dealing with the complexities of a filesystem like FAT32, so adding support for a filesystem wouldn't currently fit. (This line of MSP430, MSP430FR2433, has a maximum of 16 KB of FRAM, but perhaps I could use a different line.) 3. Photon stores images as RAW image data that require post-processing to view the images. Since I wanted to provide a polished app to view the imagery anyway, I figured I'd just make it a requirement. I'm not saying those are all particularly good reasons, but that was my reasoning anyhow.

tredre3 2y ago
It's pretty cool that your MSP handles the SD Card directly! Personally I think you should keep the focus on a polished mac app. For other platforms you can do just the bare minimum with a web app that uses WebUSB. People will bitch that it's Chromium only but, I mean, chromium has 95% market share outside Apple world (and I say this as a FF user). A hardware solution could be to add a cheap microcontroller that would handle the usb port. It wouldn't be powered by the battery, only USB and it would act as a proxy between the raw sd card and the computer. Presenting a mass storage to the computer with folders, converting images to PNGs. Such capable MCUs are very cheap (rp2040, esp32-s3) but it certainly adds complexity to your project...
- foofoo55 2y ago
  I would suggest the same hardware solution, where the USB powers a device-side presentation of the raw data into a more universal mass storage device. This also allows file transfer to any device without requiring special software. I did something like this in 2008 or so (using a FTDI chip and PIC mcu, and boy was it ever slow).
- Rinzler89 2y ago
  >It's pretty cool that your MSP handles the SD Card directly! SD cards aren't really black magic since they "speak" SPI, meaning people have been interfacing basic microcontrollers to them since forever.
  - throw46365 2y ago
    TIL!
  - LeifCarrotson 2y ago
    They speak SPI, but it's a lot slower than SDIO. And there's a lot of work that goes into making an MMC or SD card (or SDHC or SDXC card) into a FAT16 or FAT32 or exFAT filesystem! If I were doing this project, I'd use an STM32 with its built-in SDIO driver to implement most of the SD card stuff. ST's CubeIDE has easy-to-use filesystem driver that makes this transparent. Use the STM32 to read the card and manage a file for the FPGA to write the bitstream to, then when you plug into USB power, have the STM32 empty this linear file and copy the photos over to BMP images on the SD card. It would be slower, sure, but if you're only using it once a month or once a year, does that really matter? Plug it into USB, go to lunch, and come back to a properly-formatted SDXC card with an exFAT filesystem ready to have the images copied to whatever OS you want to use.
- davekeck 2y ago
  > It's pretty cool that your MSP handles the SD Card directly! The MSP430 is actually the puppet master for an ICE40 FPGA that does the high-speed transfers (102 MHz) between image sensor -> SDRAM -> SD card. Here's a post about the architecture if you're curious: https://toaster.llc/blog/architecture https://toaster.llc/blog/architecture Thanks for the feedback!
- qingcharles 2y ago
  I think open source web app with WebUSB is basically the perfect solution as others can then come in and hack a better UI onto it.
- beardyw 2y ago
  > It's pretty cool that your MSP handles the SD Card directly! No it isn't. Accessing an SD card from a microcontroller requires you to do that. To use FAT involves more effort, though lightweight libraries are available.
- barbs 2y ago
  Maybe an electron app? I mean, we all love electron, right?
Rinzler89 2y ago
Can I ask why you went with a MSP430? I'm not judging, they were all the rage in low-power embedded applications 10-15 years ago, just feels weird to see it in a modern design today, when the market now is flooded with low-power ARM chips that are way more capable than that, and probably easier to program.
- davekeck 2y ago
  One main reason I chose the MSP430 was for the FRAM. Photon stores all its state (ie the photo ring buffer indices, and user settings) in one big FRAM struct, which can be read/written just like RAM, but persists across power cycles and crashes. Another reason was MSP430's low power consumption, but like you said it sounds like ARM has caught up. Would love to hear about alternative designs -- are there low-power ARM chips that have something like FRAM and don't require erase/programming dances to write data?
  - nereye 2y ago
    If external F-RAM is an option, one can use F-RAM from, say Cypress (now Infineon, [1]) together with ARM chips from Ambiq, e.g. Apollo 3 Blue Thin [2], which likely have even better power consumption than an MSP430. [1] https://www.infineon.com/cms/en/product/memories/f-ram-ferroelectric-ram/ https://www.infineon.com/cms/en/product/memories/f-ram-ferro... [2] https://ambiq.com/apollo3-blue-thin/ https://ambiq.com/apollo3-blue-thin/
    - davekeck 2y ago
      Dang very cool. If there's ever a Photon 2 I'll definitely look at the low-power ARM world. (FWIW Photon's USB stack is handled by a STM32F730 and I've been perfectly happy with it. Expensive though.)
      - Neywiny 2y ago
        IMO that's an odd choice for a micro. It initially looked like it could translate the custom SD card writing to FAT in SW, but with only 64K of flash that's not great. Some STM HAL irqs are almost that size. My recommendation is to find 1 micro that can do low power and USB, and maybe put external FRAM/QSPI for more code storage is needed.
laweijfmvo 2y ago
Thanks for sharing the explanations, and hey it's open source so someone can make a linux version if they want!
bambax 2y ago
Ok, so an app is needed to get the data from the device to a computer, and you wanted to focus your efforts on a "polished" Mac app. But why not make a non-polished Windows app as well? Or have someone make it? Also, is the RAW data specific to the device and if so, why? If the device simply used, say, DNG or TIFF, then decoding could be done by external software, and all your own software would have to do would be to read the SD card and produce RAW files...?
- davekeck 2y ago
  > But why not make a non-polished Windows app as well? Or have someone make it? If I had more time and money I certainly would! > Also, is the RAW data specific to the device and if so, why? To maximize battery life, the hardware does zero processing on the image data; it simply copies the raw pixels output by the image sensor to the unformatted SD card. > If the device simply used, say, DNG or TIFF, then decoding could be done by external software, and all your own software would have to do would be to read the SD card and produce RAW files...? True, perhaps I should've used an existing RAW format like DNG and exposed the images as a filesystem. That would require using a different architecture (ie not using an MSP430, like Rinzler89 suggested), and would likely reduce the battery life due to having to maintain a filesystem on the SD card.
  - bambax 2y ago
    It seems it's the filesystem that's most costly? Would it be possible, maybe, to store DNG file data, one after the other, on the card, without a filesystem?
    - davekeck 2y ago
      I'm not too familiar with the DNG format, but if a DNG file could be as simple as [header][2304x1296x2 bytes][footer] then it might be possible with the current hardware. But without a filesystem, would it be useful to have DNG files stored on the SD card? Wouldn't you need special software to read off the "packed" DNG data anyway? And if you need special software anyway, couldn't the special software convert the existing raw image data to the DNG format, instead of requiring the hardware to do it?
tomcam 2y ago
Love these answers. I declare you officially to be a badass.
- hcfman 2y ago
  Indeed, kudos for that. Nice to see the out of the box thinking.
taldo 2y ago
Since the read part is handled by another chip (feeding off of USB power, therefore not terribly power constrained), you could present as a MTP device, or even mass storage, while generating the file structure on the fly. Maybe TIFFs are simple enough so you don't have to mess with the actual image bytes.
nullc 2y ago
Perhaps your perspective would be better reflected on the site if instead of just saying "exclusively for mac" it said open source right there with it-- I closed the page as "not for me" before getting down to the open source part.
LoganDark 2y ago
> To maximize Photon's battery life I tried to make the hardware do the absolute minimum to record imagery. To that end, Photon stores images on an unformatted SD card (ie one huge linear array of bytes) which is easy and fast to index into. Hmm, what about a dongle that connects between the camera and computer? The dongle wouldn't have to worry about battery life because it'd have USB power. It could do the translation to mass-storage in hardware so the computer wouldn't have to be running macOS.
airbreather 2y ago
how do you manage the balancing of card use, or is there still a card firmware layer in the loop that does that? eg lower memory being used far more often over the life so wearing out well before upper memory? how do you handle memory dropouts for a given address/cell/whatever you call a memory location, or as above?
- davekeck 2y ago
  > how do you manage the balancing of card use, or is there still a card firmware layer in the loop that does that? The SD card is accessed as a ring buffer, so all addresses are written evenly. (It was my understanding that SD cards implement wear leveling themselves though, so the wear leveling should be free even if the SD card was written to randomly -- if my understanding is correct.) > how do you handle memory dropouts for a given address/cell/whatever you call a memory location, or as above? SD card write failures cause the firmware to crash and reset itself, so it'll try again when another photo is triggered. (It also logs the reset so it can be debugged.) FWIW I'm using Samsung 128 GB "PRO Endurance" SD cards, with endurance == 820 TBW, which works out to 820e12 / 5.97e6 (size of one photo) == 137e6. So you can capture 137 million photos before the SD card is expected to fail.
bombela 2y ago
Make it a tar archive with some common RAW photo format? cat /dev/sda > raws.tar

tredre3 2y ago

It's pretty cool that your MSP handles the SD Card directly! Personally I think you should keep the focus on a polished mac app. For other platforms you can do just the bare minimum with a web app that uses WebUSB. People will bitch that it's Chromium only but, I mean, chromium has 95% market share outside Apple world (and I say this as a FF user). A hardware solution could be to add a cheap microcontroller that would handle the usb port. It wouldn't be powered by the battery, only USB and it would act as a proxy between the raw sd card and the computer. Presenting a mass storage to the computer with folders, converting images to PNGs. Such capable MCUs are very cheap (rp2040, esp32-s3) but it certainly adds complexity to your project...

foofoo55 2y ago
I would suggest the same hardware solution, where the USB powers a device-side presentation of the raw data into a more universal mass storage device. This also allows file transfer to any device without requiring special software. I did something like this in 2008 or so (using a FTDI chip and PIC mcu, and boy was it ever slow).
Rinzler89 2y ago
>It's pretty cool that your MSP handles the SD Card directly! SD cards aren't really black magic since they "speak" SPI, meaning people have been interfacing basic microcontrollers to them since forever.
- throw46365 2y ago
  TIL!
- LeifCarrotson 2y ago
  They speak SPI, but it's a lot slower than SDIO. And there's a lot of work that goes into making an MMC or SD card (or SDHC or SDXC card) into a FAT16 or FAT32 or exFAT filesystem! If I were doing this project, I'd use an STM32 with its built-in SDIO driver to implement most of the SD card stuff. ST's CubeIDE has easy-to-use filesystem driver that makes this transparent. Use the STM32 to read the card and manage a file for the FPGA to write the bitstream to, then when you plug into USB power, have the STM32 empty this linear file and copy the photos over to BMP images on the SD card. It would be slower, sure, but if you're only using it once a month or once a year, does that really matter? Plug it into USB, go to lunch, and come back to a properly-formatted SDXC card with an exFAT filesystem ready to have the images copied to whatever OS you want to use.
davekeck 2y ago
> It's pretty cool that your MSP handles the SD Card directly! The MSP430 is actually the puppet master for an ICE40 FPGA that does the high-speed transfers (102 MHz) between image sensor -> SDRAM -> SD card. Here's a post about the architecture if you're curious: https://toaster.llc/blog/architecture https://toaster.llc/blog/architecture Thanks for the feedback!
qingcharles 2y ago
I think open source web app with WebUSB is basically the perfect solution as others can then come in and hack a better UI onto it.
beardyw 2y ago
> It's pretty cool that your MSP handles the SD Card directly! No it isn't. Accessing an SD card from a microcontroller requires you to do that. To use FAT involves more effort, though lightweight libraries are available.
barbs 2y ago
Maybe an electron app? I mean, we all love electron, right?

Rinzler89 2y ago

Can I ask why you went with a MSP430? I'm not judging, they were all the rage in low-power embedded applications 10-15 years ago, just feels weird to see it in a modern design today, when the market now is flooded with low-power ARM chips that are way more capable than that, and probably easier to program.

davekeck 2y ago
One main reason I chose the MSP430 was for the FRAM. Photon stores all its state (ie the photo ring buffer indices, and user settings) in one big FRAM struct, which can be read/written just like RAM, but persists across power cycles and crashes. Another reason was MSP430's low power consumption, but like you said it sounds like ARM has caught up. Would love to hear about alternative designs -- are there low-power ARM chips that have something like FRAM and don't require erase/programming dances to write data?
- nereye 2y ago
  If external F-RAM is an option, one can use F-RAM from, say Cypress (now Infineon, [1]) together with ARM chips from Ambiq, e.g. Apollo 3 Blue Thin [2], which likely have even better power consumption than an MSP430. [1] https://www.infineon.com/cms/en/product/memories/f-ram-ferroelectric-ram/ https://www.infineon.com/cms/en/product/memories/f-ram-ferro... [2] https://ambiq.com/apollo3-blue-thin/ https://ambiq.com/apollo3-blue-thin/
  - davekeck 2y ago
    Dang very cool. If there's ever a Photon 2 I'll definitely look at the low-power ARM world. (FWIW Photon's USB stack is handled by a STM32F730 and I've been perfectly happy with it. Expensive though.)
    - Neywiny 2y ago
      IMO that's an odd choice for a micro. It initially looked like it could translate the custom SD card writing to FAT in SW, but with only 64K of flash that's not great. Some STM HAL irqs are almost that size. My recommendation is to find 1 micro that can do low power and USB, and maybe put external FRAM/QSPI for more code storage is needed.

laweijfmvo 2y ago

Thanks for sharing the explanations, and hey it's open source so someone can make a linux version if they want!

bambax 2y ago

Ok, so an app is needed to get the data from the device to a computer, and you wanted to focus your efforts on a "polished" Mac app. But why not make a non-polished Windows app as well? Or have someone make it? Also, is the RAW data specific to the device and if so, why? If the device simply used, say, DNG or TIFF, then decoding could be done by external software, and all your own software would have to do would be to read the SD card and produce RAW files...?

davekeck 2y ago
> But why not make a non-polished Windows app as well? Or have someone make it? If I had more time and money I certainly would! > Also, is the RAW data specific to the device and if so, why? To maximize battery life, the hardware does zero processing on the image data; it simply copies the raw pixels output by the image sensor to the unformatted SD card. > If the device simply used, say, DNG or TIFF, then decoding could be done by external software, and all your own software would have to do would be to read the SD card and produce RAW files...? True, perhaps I should've used an existing RAW format like DNG and exposed the images as a filesystem. That would require using a different architecture (ie not using an MSP430, like Rinzler89 suggested), and would likely reduce the battery life due to having to maintain a filesystem on the SD card.
- bambax 2y ago
  It seems it's the filesystem that's most costly? Would it be possible, maybe, to store DNG file data, one after the other, on the card, without a filesystem?
  - davekeck 2y ago
    I'm not too familiar with the DNG format, but if a DNG file could be as simple as [header][2304x1296x2 bytes][footer] then it might be possible with the current hardware. But without a filesystem, would it be useful to have DNG files stored on the SD card? Wouldn't you need special software to read off the "packed" DNG data anyway? And if you need special software anyway, couldn't the special software convert the existing raw image data to the DNG format, instead of requiring the hardware to do it?

tomcam 2y ago

Love these answers. I declare you officially to be a badass.

hcfman 2y ago
Indeed, kudos for that. Nice to see the out of the box thinking.

taldo 2y ago

Since the read part is handled by another chip (feeding off of USB power, therefore not terribly power constrained), you could present as a MTP device, or even mass storage, while generating the file structure on the fly. Maybe TIFFs are simple enough so you don't have to mess with the actual image bytes.

nullc 2y ago

Perhaps your perspective would be better reflected on the site if instead of just saying "exclusively for mac" it said open source right there with it-- I closed the page as "not for me" before getting down to the open source part.

LoganDark 2y ago

> To maximize Photon's battery life I tried to make the hardware do the absolute minimum to record imagery. To that end, Photon stores images on an unformatted SD card (ie one huge linear array of bytes) which is easy and fast to index into. Hmm, what about a dongle that connects between the camera and computer? The dongle wouldn't have to worry about battery life because it'd have USB power. It could do the translation to mass-storage in hardware so the computer wouldn't have to be running macOS.

airbreather 2y ago

how do you manage the balancing of card use, or is there still a card firmware layer in the loop that does that? eg lower memory being used far more often over the life so wearing out well before upper memory? how do you handle memory dropouts for a given address/cell/whatever you call a memory location, or as above?

davekeck 2y ago
> how do you manage the balancing of card use, or is there still a card firmware layer in the loop that does that? The SD card is accessed as a ring buffer, so all addresses are written evenly. (It was my understanding that SD cards implement wear leveling themselves though, so the wear leveling should be free even if the SD card was written to randomly -- if my understanding is correct.) > how do you handle memory dropouts for a given address/cell/whatever you call a memory location, or as above? SD card write failures cause the firmware to crash and reset itself, so it'll try again when another photo is triggered. (It also logs the reset so it can be debugged.) FWIW I'm using Samsung 128 GB "PRO Endurance" SD cards, with endurance == 820 TBW, which works out to 820e12 / 5.97e6 (size of one photo) == 137e6. So you can capture 137 million photos before the SD card is expected to fail.

bombela 2y ago

Make it a tar archive with some common RAW photo format? cat /dev/sda > raws.tar