8 ms·

Show HN: SatCat5, the open-source FPGA Ethernet switch

190 points by ooterness 3y ago

We've just launched v2.5 of SatCat5, the open-source FPGA Ethernet switch [1]. SatCat5 contains various FPGA building blocks that let you build a custom mixed-media Ethernet switch. It was originally intended for cubesats [2] but has many other potential applications. The headline feature for this release is support for the IEEE-1588 Precision Time Protocol (PTP). SatCat5 has demonstrated end-to-end synchronization to within 50 ps-rms, which is approaching the world-leading performance of CERN's White Rabbit Project [3]. Except we're doing time-transfer over regular, non-synchronous Ethernet. The key breakthrough is a new technology for digital timestamps that we've published in IEEE Access [4]. This project was featured on HN back in 2023 [5]. Since then, we've changed to the CERN-OHL-W v2.0 license, which has much better legal clarity for FPGA projects. [1] https://github.com/the-aerospace-corporation/satcat5 https://github.com/the-aerospace-corporation/satcat5 [2] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [3] https://white-rabbit.web.cern.ch/documents/White_Rabbit-a_PTP_application_for_robust_sub-nanosecond_synchronization.pdf https://white-rabbit.web.cern.ch/documents/White_Rabbit-a_PT... [4] https://ieeexplore.ieee.org/abstract/document/10367970 https://ieeexplore.ieee.org/abstract/document/10367970 [5] https://news.ycombinator.com/item?id=34817550 https://news.ycombinator.com/item?id=34817550

RantyDave 3y ago
How do you packetise stream formats (ie UART)?
- ooterness 3y ago
  SatCat5 uses the Serial Line Internet Protocol (SLIP) [1] to provide frame delimiters. Each SLIP-delimited frame contains the normal Ethernet MAC-layer header and footer (i.e., destination MAC address, source MAC address, EtherType, user data, CRC32 frame check sequence). [1] https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol
  - RantyDave 3y ago
    SLIP lives!
RantyDave 3y ago
Why Ethernet and not SpaceWire? Surely you don't need multiple hundreds of mbit/sec across a cubesat?
- ooterness 3y ago
  We continue to use SpaceWire where it's appropriate, and it was one of many options considered in the initial trade-study [1] that led to the creation of SatCat5. Among other things, many Aerospace-built cubesats have a laser communication downlink [2][3] that can reach 250 Mbps and up. It's not hard for raw sensor or video data to saturate a 1 Gbps link, and future missions may even need 10 GbE. [1] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [2] https://www.eoportal.org/satellite-missions/aerocube-ocsd https://www.eoportal.org/satellite-missions/aerocube-ocsd [3] https://aerospace.org/article/aerospaces-slingshot-1-demonstrates-pathway-accelerating-space-innovation https://aerospace.org/article/aerospaces-slingshot-1-demonst...
  - RantyDave 3y ago
    I had no idea that laser downlinks were even remotely possible from a cubesat - that's really kinda astonishing. Regarding the other high bandwidth stuff I imagined that "a communication satellite" was more or less a life support machine for a box made by Nokia. CAN is actually kinda cool, should you ever feel like going down a rabbit hole.
retrochameleon 3y ago
Interesting. Seems like it could be useful for custom ethernet protocols, eg. USB or HDMI over ethernet
minetest2048 3y ago
Hi! I'm one of the engineers working with cubesat hardware and software integration, and I've been following this project since 2023 because I agree that standardizing over Ethernet over (twisted pair, UART, single pair, etc...) will solve a lot of integration pain from all of the DIY protocols that the cubesat/smallsat community uses Some thoughts: - You actually have a publicly available ICD at https://github.com/the-aerospace-corporation/satcat5/tree/main/examples/slingshot https://github.com/the-aerospace-corporation/satcat5/tree/ma... , please publicize this more in the readme! You're a very rare exception in a world of contact us and sign an NDA first and commercial in confidence ICDs, which is great, and leads me to my next point: - Compared to HTTP API documentation, space ICDs sucks. One of the ICDs that I have put its protocol documentation as screenshots instead of copy pasteable text. Of course ICDs sucks because you won't know its quality until you buy it... Anyway standardizing on ethernet for transport layer give you all the tooling at 99% of the world's computer use, and it might be a first step in making a OpenAPI / Swagger type tool but for space protocols - Any plans for 10base-T1S single pair ethernet? Our 6U cubesats barely have enough power to run its payload computers, let alone a FPGA based switch. But in a way a switched star network can be more reliable than a multidrop bus, as you can isolate babbling idiot nodes, and filter out accidental/intentional MAC/IP address spoofing
- ooterness 3y ago
  Thanks for a very thoughtful reply! Regarding the Slingshot ICD: Noted, I'll make a point to update the readme. That document is literally the payload ICD for the Slingshot-1 cubesat that launched in July 2022 [1][2]. Future missions may take things in a slightly different direction, but that ICD is a flight-proven point design that shows what's possible. The space industry's NDA-by-default attitude definitely creates a lot of unnecessary friction. I totally agree that off-the-shelf tooling is a huge benefit of leaning into Ethernet. In particular, Wireshark has been essential in winning over hesitant developers; it's just light-years ahead of what you usually get for debugging microcontrollers. I've been watching the various 10BASE-T1S / 100BASE-T1 / 1000BASE-T1 standards, but company interest has been limited. AFAIK none of them can be attached directly to FPGA GPIO/SERDES/etc., so you need a separate PHY chip on both ends of the link. For us, the competition is SGMII over coax (for FPGA payloads) or SPI/UART over 4 wires (for microcontroller payloads). Cubesat cable runs are so short that the ability to directly attach to GPIO typically outweighs potential size and weight savings in the cable harness, plus it's one less chip with unknown radiation tolerance, etc. That said, the good news is that -T1 PHYs typically have one of the usual *MII options (RMII, SGMII, etc.), so many are already compatible with SatCat5. The bad news is 10BASE-T1S requires CS/MA and PLCA logic that is not yet supported. (But if anyone wants to contribute cool features to this open source project, please reach out.) [1] https://aerospace.org/sites/default/files/2022-08/Slingshot_0822.pdf https://aerospace.org/sites/default/files/2022-08/Slingshot_... [2] https://aerospace.org/article/aerospaces-slingshot-1-demonstrates-pathway-accelerating-space-innovation https://aerospace.org/article/aerospaces-slingshot-1-demonst...
crote 3y ago
Huh, what a neat project! Especially the Ethernet-over-UART/SPI/whatever part is quite interesting. It's incredibly obvious in hindsight, but it's of course a great way to create affordable wired IoT devices. You could just talk Ethernet over RS485, and be able to communicate to a bunch of devices with a single $20 translator and a $0.15 transceiver per device!
- ooterness 3y ago
  Yes, that's a great example of the "mixed-media" use case for SatCat5. If you have an Ethernet switch where some ports are Ethernet-over-RS485, and others are Ethernet-over-10/100/1000BASE-T, then the switch doubles as a sort of media converter. Using a common transport layer means it's easier to link sensors, actuators, and compute resources.
wrs 3y ago
Maybe I’m just looking in the wrong places, but open hardware at this quality level seems to be extremely rare. Does anyone have other good examples of complete, professional-quality open source HDL projects that aren’t CPUs? I’m thinking of the hardware equivalent of PostgreSQL, something truly industrial strength with readable code you can learn from.
- ooterness 3y ago
  The open-source community for gateware is not as large or well-developed as the one for software, but there's many good projects out there. OpenCores [1] has been around a long time, and there's many high-quality FPGA IP-cores hosted there. The Open Hardware Repository [2] is another good resource. Not all OHWR projects involve FPGAs, but many have full hardware/gateware/software stacks, including the White Rabbit Project [3]. [1] https://opencores.org/ https://opencores.org/ [2] https://ohwr.org/explore/projects/starred https://ohwr.org/explore/projects/starred [3] https://white-rabbit.web.cern.ch/ https://white-rabbit.web.cern.ch/
  - wrs 3y ago
    I know about OpenCores, but it consists of, well, cores. I’m looking for fully realized projects, with controller, glue logic, external interfaces, test benches, multiple hardware realizations, all the tedious stuff you have to do to make a working version>1 device. That is, what do you do with the cores? To me it seems analogous to the difference between a B-tree library and PostgreSQL. White Rabbit looks great, thanks for the pointer!
    - mng2 3y ago
      As the OP says, FPGAs are kind of niche and devoted to specialized applications. For example there's Corundum[0] for in-network compute, kind of exotic and maybe too pricey for a hobbyist. The biggest open-source thing out there is of course MiSTer, but it is tied to a particular old dev board and toolchain last I checked. >what do you _do_ with the cores? I mean, this is kind of the problem, but in a way it's a funny problem. The strength and the curse of FPGAs is that they can become almost anything you want. However getting that sea of logic into the desired form requires knowledge of the FPGA family, the tools, the hardware, and digital design in general. The "tedious stuff" unfortunately makes up a great deal of FPGA work! Many days I feel like a glorified plumber... As far as cores go, in FPGA land, the vendor would probably already provide a core (IP, in hardware parlance) for say fooing bars, that is easier to use and probably a far sight more reliable than the project on OpenCores. (Which probably hasn't been touched in a decade, is written in VHDL-87, is using Wishbone when you want AXI, and lacks a test suite.) As a hobbyist, you might search and find the project on OpenCores, but be scared away by the cruftiness. You then write your own foo_bars module, thereby learning a great deal, but contributing to the fragmentation problem. Plus seeding the internet with weird code with corner-case bugs for LLMs to trip over. [0]: https://github.com/corundum/corundum https://github.com/corundum/corundum
shmerl 3y ago
Oh, is there an open source switch that supports 10 Gbps SFP+?
- morphle 3y ago
  I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28. Older model described in [1]. Email me if you want one. I also sell cheap $150 10 Gbps and 25 Gbps QSFP28 switches but they are only partially open source. They still sell some of the NetFPGA models, very pricy [2]. There are a few Chinese and Russian switches but you can't buy them anymore because of the war. [1] https://www.researchgate.net/profile/Merik-Voswinkel/publication/309254511_Fiberhood_Smart_Grid/links/5807410c08ae5ad1881691e3/Fiberhood-Smart-Grid.pdf https://www.researchgate.net/profile/Merik-Voswinkel/publica... [2] https://netfpga.org https://netfpga.org
  - shmerl 3y ago
    > I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28. How big is it physically? I don't really get it from the link.
    - morphle 3y ago
      There is a picture of a FPGA switch in the pdf [1]. You usually put it in a outdoor or indoor metal case or a PC slot.
- ooterness 3y ago
  Development of SFP+/XAUI is on our wishlist for SatCat5, but not yet supported.
gigatexal 3y ago
In my next life I will go to college an study computer engineering. It never ceases to amaze me the cool stuff you hardware hackers can do.
zokier 3y ago
One of my dream projects would be to grab some off-the-shelf switch IC and build OSHW switch around it with fully free blobless Linux based software stack. Something like SparX-5 [1] could work here, and the vendor software is already Linux based (but presumably absolutely horrible). The problem is that you'd need fair chunk of cash to get anywhere, so this basically needs a kickstarter or something to get going. [1] https://www.microchip.com/en-us/product/vsc7552 https://www.microchip.com/en-us/product/vsc7552
- tonyarkles 3y ago
  You might be interested in some of the stuff Andrew Zonenberg is working on: https://ioc.exchange/@Azonenberg https://ioc.exchange/@Azonenberg. I don’t have a link to a specific post, he usually has a couple of things on the go and they all get interleaved. Pretty wild Ethernet stuff in there!

190 points by ooterness 3y ago

RantyDave 3y ago

How do you packetise stream formats (ie UART)?

ooterness 3y ago
SatCat5 uses the Serial Line Internet Protocol (SLIP) [1] to provide frame delimiters. Each SLIP-delimited frame contains the normal Ethernet MAC-layer header and footer (i.e., destination MAC address, source MAC address, EtherType, user data, CRC32 frame check sequence). [1] https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol
- RantyDave 3y ago
  SLIP lives!

RantyDave 3y ago

Why Ethernet and not SpaceWire? Surely you don't need multiple hundreds of mbit/sec across a cubesat?

ooterness 3y ago
We continue to use SpaceWire where it's appropriate, and it was one of many options considered in the initial trade-study [1] that led to the creation of SatCat5. Among other things, many Aerospace-built cubesats have a laser communication downlink [2][3] that can reach 250 Mbps and up. It's not hard for raw sensor or video data to saturate a 1 Gbps link, and future missions may even need 10 GbE. [1] https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ https://digitalcommons.usu.edu/smallsat/2020/all2020/174/ [2] https://www.eoportal.org/satellite-missions/aerocube-ocsd https://www.eoportal.org/satellite-missions/aerocube-ocsd [3] https://aerospace.org/article/aerospaces-slingshot-1-demonstrates-pathway-accelerating-space-innovation https://aerospace.org/article/aerospaces-slingshot-1-demonst...
- RantyDave 3y ago
  I had no idea that laser downlinks were even remotely possible from a cubesat - that's really kinda astonishing. Regarding the other high bandwidth stuff I imagined that "a communication satellite" was more or less a life support machine for a box made by Nokia. CAN is actually kinda cool, should you ever feel like going down a rabbit hole.

retrochameleon 3y ago

Interesting. Seems like it could be useful for custom ethernet protocols, eg. USB or HDMI over ethernet

minetest2048 3y ago

Hi! I'm one of the engineers working with cubesat hardware and software integration, and I've been following this project since 2023 because I agree that standardizing over Ethernet over (twisted pair, UART, single pair, etc...) will solve a lot of integration pain from all of the DIY protocols that the cubesat/smallsat community uses Some thoughts: - You actually have a publicly available ICD at https://github.com/the-aerospace-corporation/satcat5/tree/main/examples/slingshot https://github.com/the-aerospace-corporation/satcat5/tree/ma... , please publicize this more in the readme! You're a very rare exception in a world of contact us and sign an NDA first and commercial in confidence ICDs, which is great, and leads me to my next point: - Compared to HTTP API documentation, space ICDs sucks. One of the ICDs that I have put its protocol documentation as screenshots instead of copy pasteable text. Of course ICDs sucks because you won't know its quality until you buy it... Anyway standardizing on ethernet for transport layer give you all the tooling at 99% of the world's computer use, and it might be a first step in making a OpenAPI / Swagger type tool but for space protocols - Any plans for 10base-T1S single pair ethernet? Our 6U cubesats barely have enough power to run its payload computers, let alone a FPGA based switch. But in a way a switched star network can be more reliable than a multidrop bus, as you can isolate babbling idiot nodes, and filter out accidental/intentional MAC/IP address spoofing

ooterness 3y ago
Thanks for a very thoughtful reply! Regarding the Slingshot ICD: Noted, I'll make a point to update the readme. That document is literally the payload ICD for the Slingshot-1 cubesat that launched in July 2022 [1][2]. Future missions may take things in a slightly different direction, but that ICD is a flight-proven point design that shows what's possible. The space industry's NDA-by-default attitude definitely creates a lot of unnecessary friction. I totally agree that off-the-shelf tooling is a huge benefit of leaning into Ethernet. In particular, Wireshark has been essential in winning over hesitant developers; it's just light-years ahead of what you usually get for debugging microcontrollers. I've been watching the various 10BASE-T1S / 100BASE-T1 / 1000BASE-T1 standards, but company interest has been limited. AFAIK none of them can be attached directly to FPGA GPIO/SERDES/etc., so you need a separate PHY chip on both ends of the link. For us, the competition is SGMII over coax (for FPGA payloads) or SPI/UART over 4 wires (for microcontroller payloads). Cubesat cable runs are so short that the ability to directly attach to GPIO typically outweighs potential size and weight savings in the cable harness, plus it's one less chip with unknown radiation tolerance, etc. That said, the good news is that -T1 PHYs typically have one of the usual *MII options (RMII, SGMII, etc.), so many are already compatible with SatCat5. The bad news is 10BASE-T1S requires CS/MA and PLCA logic that is not yet supported. (But if anyone wants to contribute cool features to this open source project, please reach out.) [1] https://aerospace.org/sites/default/files/2022-08/Slingshot_0822.pdf https://aerospace.org/sites/default/files/2022-08/Slingshot_... [2] https://aerospace.org/article/aerospaces-slingshot-1-demonstrates-pathway-accelerating-space-innovation https://aerospace.org/article/aerospaces-slingshot-1-demonst...

crote 3y ago

Huh, what a neat project! Especially the Ethernet-over-UART/SPI/whatever part is quite interesting. It's incredibly obvious in hindsight, but it's of course a great way to create affordable wired IoT devices. You could just talk Ethernet over RS485, and be able to communicate to a bunch of devices with a single $20 translator and a $0.15 transceiver per device!

ooterness 3y ago
Yes, that's a great example of the "mixed-media" use case for SatCat5. If you have an Ethernet switch where some ports are Ethernet-over-RS485, and others are Ethernet-over-10/100/1000BASE-T, then the switch doubles as a sort of media converter. Using a common transport layer means it's easier to link sensors, actuators, and compute resources.

wrs 3y ago

Maybe I’m just looking in the wrong places, but open hardware at this quality level seems to be extremely rare. Does anyone have other good examples of complete, professional-quality open source HDL projects that aren’t CPUs? I’m thinking of the hardware equivalent of PostgreSQL, something truly industrial strength with readable code you can learn from.

ooterness 3y ago
The open-source community for gateware is not as large or well-developed as the one for software, but there's many good projects out there. OpenCores [1] has been around a long time, and there's many high-quality FPGA IP-cores hosted there. The Open Hardware Repository [2] is another good resource. Not all OHWR projects involve FPGAs, but many have full hardware/gateware/software stacks, including the White Rabbit Project [3]. [1] https://opencores.org/ https://opencores.org/ [2] https://ohwr.org/explore/projects/starred https://ohwr.org/explore/projects/starred [3] https://white-rabbit.web.cern.ch/ https://white-rabbit.web.cern.ch/
- wrs 3y ago
  I know about OpenCores, but it consists of, well, cores. I’m looking for fully realized projects, with controller, glue logic, external interfaces, test benches, multiple hardware realizations, all the tedious stuff you have to do to make a working version>1 device. That is, what do you do with the cores? To me it seems analogous to the difference between a B-tree library and PostgreSQL. White Rabbit looks great, thanks for the pointer!
  - mng2 3y ago
    As the OP says, FPGAs are kind of niche and devoted to specialized applications. For example there's Corundum[0] for in-network compute, kind of exotic and maybe too pricey for a hobbyist. The biggest open-source thing out there is of course MiSTer, but it is tied to a particular old dev board and toolchain last I checked. >what do you _do_ with the cores? I mean, this is kind of the problem, but in a way it's a funny problem. The strength and the curse of FPGAs is that they can become almost anything you want. However getting that sea of logic into the desired form requires knowledge of the FPGA family, the tools, the hardware, and digital design in general. The "tedious stuff" unfortunately makes up a great deal of FPGA work! Many days I feel like a glorified plumber... As far as cores go, in FPGA land, the vendor would probably already provide a core (IP, in hardware parlance) for say fooing bars, that is easier to use and probably a far sight more reliable than the project on OpenCores. (Which probably hasn't been touched in a decade, is written in VHDL-87, is using Wishbone when you want AXI, and lacks a test suite.) As a hobbyist, you might search and find the project on OpenCores, but be scared away by the cruftiness. You then write your own foo_bars module, thereby learning a great deal, but contributing to the fragmentation problem. Plus seeding the internet with weird code with corner-case bugs for LLMs to trip over. [0]: https://github.com/corundum/corundum https://github.com/corundum/corundum

shmerl 3y ago

Oh, is there an open source switch that supports 10 Gbps SFP+?

morphle 3y ago
I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28. Older model described in [1]. Email me if you want one. I also sell cheap $150 10 Gbps and 25 Gbps QSFP28 switches but they are only partially open source. They still sell some of the NetFPGA models, very pricy [2]. There are a few Chinese and Russian switches but you can't buy them anymore because of the war. [1] https://www.researchgate.net/profile/Merik-Voswinkel/publication/309254511_Fiberhood_Smart_Grid/links/5807410c08ae5ad1881691e3/Fiberhood-Smart-Grid.pdf https://www.researchgate.net/profile/Merik-Voswinkel/publica... [2] https://netfpga.org https://netfpga.org
- shmerl 3y ago
  > I make my own open source FPGA switches, 6 and 12 port 10 Gbps SFP+ up to 100 Gbps QSFP28. How big is it physically? I don't really get it from the link.
  - morphle 3y ago
    There is a picture of a FPGA switch in the pdf [1]. You usually put it in a outdoor or indoor metal case or a PC slot.
ooterness 3y ago
Development of SFP+/XAUI is on our wishlist for SatCat5, but not yet supported.

gigatexal 3y ago

In my next life I will go to college an study computer engineering. It never ceases to amaze me the cool stuff you hardware hackers can do.

zokier 3y ago

One of my dream projects would be to grab some off-the-shelf switch IC and build OSHW switch around it with fully free blobless Linux based software stack. Something like SparX-5 [1] could work here, and the vendor software is already Linux based (but presumably absolutely horrible). The problem is that you'd need fair chunk of cash to get anywhere, so this basically needs a kickstarter or something to get going. [1] https://www.microchip.com/en-us/product/vsc7552 https://www.microchip.com/en-us/product/vsc7552

tonyarkles 3y ago
You might be interested in some of the stuff Andrew Zonenberg is working on: https://ioc.exchange/@Azonenberg https://ioc.exchange/@Azonenberg. I don’t have a link to a specific post, he usually has a couple of things on the go and they all get interleaved. Pretty wild Ethernet stuff in there!