10 ms·

LibreRPi – open source replacements for RPi firmware

315 points by pabs3 4y ago

ggm 4y ago
A probably naieve question. If the VPU can be used as the bootstrap loader for the CPU, much as a PDP11 on the door of the Vax 11/780 acted as bootstrap loader for the Vax, can we not simply replace all the complex VPU resident functions with 'just run on the CPU' and then resolve the external device bindings in normal CPU code? Maybe I misread it, but it read like the VPU does more than the tiny necessary bootstrap to find an image, and chain load it. It reads like it has to do some timing critical initialisations of subsystems, but I cannot but think a CPU should be able to do this for itself, once it has some run state with a kernel in memory. It's as if the VPU has to make the other bits of blobby hardware be alive, for the CPU to be able to see them. Most odd. Oftentimes there are magic bits of this to do with ring states, and priviledge to alter ring at runtime. Maybe this is part of the story: the VPU initialises a specific ring state and some outer higher trust ring MUST be pre-initialised for the CPU ring state to be allowed to talk to it: it would violate the ring model, if the CPU did it.
- mburee 4y ago
  https://github.com/librerpi/rpi-open-firmware https://github.com/librerpi/rpi-open-firmware The paragraph 'Does it boot Linux' should answer your questions
  - ggm 4y ago
    Well no. It says: Yes, with some conditions. You can boot a very minimal version of Linux without the firmware and get it to work with UART and eMMC. Support for USB, DMA, and Ethernet are in the works, which will be sufficient for certain headless systems. Still, many other peripherals require bringup, such as video. Additionally, drivers for power management need to written. So it isn't clear if this is code in the kernel, or code blobs still to be run in the VPU, and why. "drivers" could refer to linux kernel, or the drivers as blob downloads, or VPU initialization.
    - pabs3 4y ago
      This is separate code (a fork of the Little Kernel project) that runs on the VPU and loads later stages of the boot process, including initialising the ARM core and running the Linux kernel on it. https://github.com/librerpi/lk-overlay https://github.com/librerpi/lk-overlay Edit: The rpi-firmware repo has been replaced by the lk-overlay repo, which is much more flexible.
- pjc50 4y ago
  I'm not aware of the specific details, but one of the usual critical things is clock and PLL configuration. It looks like the PLLs that generate all the internal clock frequencies, including the 48MHz for USB, are connected to the VPU. It may be the case that relevant hardware simply isn't connected to the ARM. (From Broadcom's point of view, the VPU is the "real" computer with all the processing power, and the ARM is tacked on the side)
  - ggm 4y ago
    This is so true, and so amusing. It almost makes me wonder if the microinstructions of the VPU could be where we run.. and use this ARM chip to do things like AES offload! Thanks for a very useful observation. Clocking is fundamental to a lot of things, buses like the clock to be done right so I can believe booting requires good clocking and some bus initialisation.
    - pjc50 4y ago
      Turns out there is some pretty good architecture documentation: https://docs.broadcom.com/doc/12358545 https://docs.broadcom.com/doc/12358545 .. but that only describes the 3D capabilities, not the whole register map.
      - q-big 4y ago
        > Turns out there is some pretty good architecture documentation: https://docs.broadcom.com/doc/12358545 https://docs.broadcom.com/doc/12358545 .. but that only describes the 3D capabilities, not the whole register map. This documentation is for the VideoCore IV. The BCM2711 SOC that is used in the Raspberry Pi 4 has a VideoCore VI.
      - monocasa 4y ago
        That doesn't really cover the processor under discussion here, only the shader cores called the QPUs. This work and the processor under discussion is another core called the VPU that unfortunately gets confused because with the default firmware it runs the majority of the OpenGL driver stack, controlling the QPUs.
- nerdponx 4y ago
  Fun fact, the "ï" in "naïve" is not an umlaut, but a totally different use of the same notation called a diaresis! It's meant to inform the reader that the "a" and "i" are pronounced separately, rather than being combined into a single vowel sound. It's a bit old-fashioned in English, but you sometimes will see it in older texts on words like "coöperation". So "naieve" is not a valid spelling of "naïve" (but that's okay because English spelling has historically been a mess and everyone knew what you meant). https://en.m.wikipedia.org/wiki/Diaeresis_(diacritic) https://en.m.wikipedia.org/wiki/Diaeresis_(diacritic)
  - tomcam 4y ago
    I just had diaresis. Painful at my age.
    - boredemployee 4y ago
      You should check your vitamin D levels
      - tomcam 4y ago
        Username checks out gloriously for assisting with my drastic medical condition
        boredemployee 4y ago
        lol we should unite
    - Inhibit 4y ago
      More fiber and less umlauts for breakfast 'll fix that.
      - benj111 4y ago
        An Oxford comma a day keeps the diaresis at bay.
- cleverca22 4y ago
  some peripherals like the dram controller are rigged up so the arm can never control them others like the 2d core default to the same mode, but can be re-configured to allow the arm to control it there may be more peripherals that behave the same way, and require code on the VPU to help out and like others said, changing the arm freq, from the arm core, could be dangerous, it may just lock up, because it turned its own clock off
- xani_ 4y ago
  IIRC it was just architecture quirk of those chips as initally they were designed as basically fancy accelerators to be used with already existing systems > VideoCore chips can run complete applications - they are not simply video DSP chips that require a separate processor to supply and collect data. In practice though, they are often used like simple accelerators, as companies usually prefer to cautiously assimilate new technology rather than take a big risk in porting a large amount of application code from an existing ARM-based design. For example one of earlier ones was used as GPU for Nokia N8
pabs3 4y ago
A post about the status of this project from earlier this year: https://gwolf.org/2022/04/how-is-the-free-firmware-for-the-raspberry-progressing.html https://gwolf.org/2022/04/how-is-the-free-firmware-for-the-r... https://news.ycombinator.com/item?id=31000725 https://news.ycombinator.com/item?id=31000725
- xani_ 4y ago
  Huh, didn't knew rPi foundation purposefully added DRM to their products...WTF ?
pabs3 4y ago
I encourage anyone who is interested in free boot/other firmware for the RPi to get involved in the project, it is mostly one person at the moment (not me).
pabs3 4y ago
The most recent work for this project is in the lk-overlay repo, which is an overlay of the Little Kernel embedded OS project: https://github.com/librerpi/lk-overlay https://github.com/librerpi/lk-overlay
RobotToaster 4y ago
I guess this begs the question, are there any RPi alternatives/equivalents that use an open source firmware by default?
- tpxl 4y ago
  The Pine64 guys are working towards that goal I think. You can flash open source firmware on some of their boards, but I don't know if it's the default.
  - deleted 4y ago
    [deleted]
- egwynn 4y ago
  I’m not an expert, but iirc banana-pi and beagleboard are SBCs with more of an open source bent.
- marcodiego 4y ago
  Some RK3399 boards can be used AFAIK entirely blob free. This has been true for a few years actually: https://stikonas.eu/wordpress/2019/09/15/blobless-boot-with-rockpro64/ https://stikonas.eu/wordpress/2019/09/15/blobless-boot-with-...
- megous 4y ago
  Pretty much all Allwinner based SoCs used in common SBCs have FOSS firmware. Some older Rochcip SoCs also do, but you have to be careful there (the most open/fully featured is RK3399). That covers a huge amount of SBCs out there. Some even have multiple alternatives for FOSS firmware. If anything RPi seems like an outlier. Out of 9 different SBCs models I have at home only Rpi2b and Quartz64-A have proprietary firmware.
- kevin_thibedeau 4y ago
  Beaglebone
actionfromafar 4y ago
This project focuses on being a drop in replacement to the proprietary firmware needed to bringup Linux on the other processor. (The ARM processor.) Isn't anyone going to talk about running random stuff on the VPU itself?! Given it's the thing used for 3D and stuff, I imagine it's going to be crazy fast at certain computations, much faster than the ARM is. Untapped potential of all Raspberry Pies out there... maybe the VPU can be (ab)used to do stuff similar to what the Raspberry Pico can do.
- cleverca22 4y ago
  yep, thats why ive been focusing a lot on porting linux drivers over to little-kernel on the vpu, and getting things working without linux https://www.youtube.com/watch?v=l7lIewA9fm4 https://www.youtube.com/watch?v=l7lIewA9fm4 is an example of a VPU accelerated mandelbrot, without the arm core even being turned on for (int i=0; i<16; i++) { int temp = a[i] * b[i]; if (store) c[i] = temp; if (accumulate) accumulator[i] += temp; } basically, the VPU can run this entire line of code in just 2 clock cycles, and this isnt even the 3d core, which is entirely seperate and more complex
marcodiego 4y ago
I thought this project was dead. It is good to see some progress on this.
- pabs3 4y ago
  I guess you are thinking of this issue: https://github.com/christinaa/rpi-open-firmware/issues/37 https://github.com/christinaa/rpi-open-firmware/issues/37 Since then the project moved to a new maintainer (not me), who worked on it slowly but surely. They need new contributors though.
Thoreandan 4y ago
So fully-functional blob-free RPi3 / RPi4 is a thing now?
- pabs3 4y ago
  Not yet, but it could be if more people help contribute to it.
mmastrac 4y ago
I really hope they can get HDMI up on this because I'm struggling with an incompatibility between my IP KVM and Raspberry Pis. I would really love to dig into the HDMI initialization code. Something about the HDMI output on a Pi is just absolutely incompatible with the IP KVM's interface modules and it's maddeningly subtle because the Pis work _everywhere_ else.
grokx 4y ago
They should have called it "LiberPi"! :)
lproven 4y ago
Nobody's mentioned it yet, so... One reason this is so difficult is that the VPU runs an entire standalone OS of its own. AIUI, the VPU is in control of the computer, and the Arm is an accessory device. Only once the VPU has loaded its OS and it's running will the Arm be initialised, the kernel image loaded into RAM by the VPU, and then the Arm pointed at it. The OS continues to run and control the computer even once the Arm is running. The VPU runs a proprietary RTOS called ThreadX, which is now owned by Microsoft. https://en.wikipedia.org/wiki/ThreadX https://en.wikipedia.org/wiki/ThreadX
- cleverca22 4y ago
  the rpi-open-firmware codebase is going the route of doing as little as possible on the VPU, and giving the arm control fairly early i found that a bit limiting, with the lack of threads/drivers/modularity thats why i ported little-kernel, to get the same level of features threadx likely offered, and then be able to expand the feature-set far more with the lk-overlay codebase, you can drive the 2d and 3d cores from the VPU you can output ntsc or vga video you can turn on the arm core and boot linux and you can map /dev/fb0 on the video output and there are various cpu-independent features in LK like tga image decoding

315 points by pabs3 4y ago

ggm 4y ago

A probably naieve question. If the VPU can be used as the bootstrap loader for the CPU, much as a PDP11 on the door of the Vax 11/780 acted as bootstrap loader for the Vax, can we not simply replace all the complex VPU resident functions with 'just run on the CPU' and then resolve the external device bindings in normal CPU code? Maybe I misread it, but it read like the VPU does more than the tiny necessary bootstrap to find an image, and chain load it. It reads like it has to do some timing critical initialisations of subsystems, but I cannot but think a CPU should be able to do this for itself, once it has some run state with a kernel in memory. It's as if the VPU has to make the other bits of blobby hardware be alive, for the CPU to be able to see them. Most odd. Oftentimes there are magic bits of this to do with ring states, and priviledge to alter ring at runtime. Maybe this is part of the story: the VPU initialises a specific ring state and some outer higher trust ring MUST be pre-initialised for the CPU ring state to be allowed to talk to it: it would violate the ring model, if the CPU did it.

mburee 4y ago
https://github.com/librerpi/rpi-open-firmware https://github.com/librerpi/rpi-open-firmware The paragraph 'Does it boot Linux' should answer your questions
- ggm 4y ago
  Well no. It says: Yes, with some conditions. You can boot a very minimal version of Linux without the firmware and get it to work with UART and eMMC. Support for USB, DMA, and Ethernet are in the works, which will be sufficient for certain headless systems. Still, many other peripherals require bringup, such as video. Additionally, drivers for power management need to written. So it isn't clear if this is code in the kernel, or code blobs still to be run in the VPU, and why. "drivers" could refer to linux kernel, or the drivers as blob downloads, or VPU initialization.
  - pabs3 4y ago
    This is separate code (a fork of the Little Kernel project) that runs on the VPU and loads later stages of the boot process, including initialising the ARM core and running the Linux kernel on it. https://github.com/librerpi/lk-overlay https://github.com/librerpi/lk-overlay Edit: The rpi-firmware repo has been replaced by the lk-overlay repo, which is much more flexible.
pjc50 4y ago
I'm not aware of the specific details, but one of the usual critical things is clock and PLL configuration. It looks like the PLLs that generate all the internal clock frequencies, including the 48MHz for USB, are connected to the VPU. It may be the case that relevant hardware simply isn't connected to the ARM. (From Broadcom's point of view, the VPU is the "real" computer with all the processing power, and the ARM is tacked on the side)
- ggm 4y ago
  This is so true, and so amusing. It almost makes me wonder if the microinstructions of the VPU could be where we run.. and use this ARM chip to do things like AES offload! Thanks for a very useful observation. Clocking is fundamental to a lot of things, buses like the clock to be done right so I can believe booting requires good clocking and some bus initialisation.
  - pjc50 4y ago
    Turns out there is some pretty good architecture documentation: https://docs.broadcom.com/doc/12358545 https://docs.broadcom.com/doc/12358545 .. but that only describes the 3D capabilities, not the whole register map.
    - q-big 4y ago
      > Turns out there is some pretty good architecture documentation: https://docs.broadcom.com/doc/12358545 https://docs.broadcom.com/doc/12358545 .. but that only describes the 3D capabilities, not the whole register map. This documentation is for the VideoCore IV. The BCM2711 SOC that is used in the Raspberry Pi 4 has a VideoCore VI.
    - monocasa 4y ago
      That doesn't really cover the processor under discussion here, only the shader cores called the QPUs. This work and the processor under discussion is another core called the VPU that unfortunately gets confused because with the default firmware it runs the majority of the OpenGL driver stack, controlling the QPUs.
nerdponx 4y ago
Fun fact, the "ï" in "naïve" is not an umlaut, but a totally different use of the same notation called a diaresis! It's meant to inform the reader that the "a" and "i" are pronounced separately, rather than being combined into a single vowel sound. It's a bit old-fashioned in English, but you sometimes will see it in older texts on words like "coöperation". So "naieve" is not a valid spelling of "naïve" (but that's okay because English spelling has historically been a mess and everyone knew what you meant). https://en.m.wikipedia.org/wiki/Diaeresis_(diacritic) https://en.m.wikipedia.org/wiki/Diaeresis_(diacritic)
- tomcam 4y ago
  I just had diaresis. Painful at my age.
  - boredemployee 4y ago
    You should check your vitamin D levels
    - tomcam 4y ago
      Username checks out gloriously for assisting with my drastic medical condition
      - boredemployee 4y ago
        lol we should unite
  - Inhibit 4y ago
    More fiber and less umlauts for breakfast 'll fix that.
    - benj111 4y ago
      An Oxford comma a day keeps the diaresis at bay.
cleverca22 4y ago
some peripherals like the dram controller are rigged up so the arm can never control them others like the 2d core default to the same mode, but can be re-configured to allow the arm to control it there may be more peripherals that behave the same way, and require code on the VPU to help out and like others said, changing the arm freq, from the arm core, could be dangerous, it may just lock up, because it turned its own clock off
xani_ 4y ago
IIRC it was just architecture quirk of those chips as initally they were designed as basically fancy accelerators to be used with already existing systems > VideoCore chips can run complete applications - they are not simply video DSP chips that require a separate processor to supply and collect data. In practice though, they are often used like simple accelerators, as companies usually prefer to cautiously assimilate new technology rather than take a big risk in porting a large amount of application code from an existing ARM-based design. For example one of earlier ones was used as GPU for Nokia N8

pabs3 4y ago

A post about the status of this project from earlier this year: https://gwolf.org/2022/04/how-is-the-free-firmware-for-the-raspberry-progressing.html https://gwolf.org/2022/04/how-is-the-free-firmware-for-the-r... https://news.ycombinator.com/item?id=31000725 https://news.ycombinator.com/item?id=31000725

xani_ 4y ago
Huh, didn't knew rPi foundation purposefully added DRM to their products...WTF ?

pabs3 4y ago

I encourage anyone who is interested in free boot/other firmware for the RPi to get involved in the project, it is mostly one person at the moment (not me).

pabs3 4y ago

The most recent work for this project is in the lk-overlay repo, which is an overlay of the Little Kernel embedded OS project: https://github.com/librerpi/lk-overlay https://github.com/librerpi/lk-overlay

RobotToaster 4y ago

I guess this begs the question, are there any RPi alternatives/equivalents that use an open source firmware by default?

tpxl 4y ago
The Pine64 guys are working towards that goal I think. You can flash open source firmware on some of their boards, but I don't know if it's the default.
- deleted 4y ago
  [deleted]
egwynn 4y ago
I’m not an expert, but iirc banana-pi and beagleboard are SBCs with more of an open source bent.
marcodiego 4y ago
Some RK3399 boards can be used AFAIK entirely blob free. This has been true for a few years actually: https://stikonas.eu/wordpress/2019/09/15/blobless-boot-with-rockpro64/ https://stikonas.eu/wordpress/2019/09/15/blobless-boot-with-...
megous 4y ago
Pretty much all Allwinner based SoCs used in common SBCs have FOSS firmware. Some older Rochcip SoCs also do, but you have to be careful there (the most open/fully featured is RK3399). That covers a huge amount of SBCs out there. Some even have multiple alternatives for FOSS firmware. If anything RPi seems like an outlier. Out of 9 different SBCs models I have at home only Rpi2b and Quartz64-A have proprietary firmware.
kevin_thibedeau 4y ago
Beaglebone

actionfromafar 4y ago

This project focuses on being a drop in replacement to the proprietary firmware needed to bringup Linux on the other processor. (The ARM processor.) Isn't anyone going to talk about running random stuff on the VPU itself?! Given it's the thing used for 3D and stuff, I imagine it's going to be crazy fast at certain computations, much faster than the ARM is. Untapped potential of all Raspberry Pies out there... maybe the VPU can be (ab)used to do stuff similar to what the Raspberry Pico can do.

cleverca22 4y ago
yep, thats why ive been focusing a lot on porting linux drivers over to little-kernel on the vpu, and getting things working without linux https://www.youtube.com/watch?v=l7lIewA9fm4 https://www.youtube.com/watch?v=l7lIewA9fm4 is an example of a VPU accelerated mandelbrot, without the arm core even being turned on for (int i=0; i<16; i++) { int temp = a[i] * b[i]; if (store) c[i] = temp; if (accumulate) accumulator[i] += temp; } basically, the VPU can run this entire line of code in just 2 clock cycles, and this isnt even the 3d core, which is entirely seperate and more complex

marcodiego 4y ago

I thought this project was dead. It is good to see some progress on this.

pabs3 4y ago
I guess you are thinking of this issue: https://github.com/christinaa/rpi-open-firmware/issues/37 https://github.com/christinaa/rpi-open-firmware/issues/37 Since then the project moved to a new maintainer (not me), who worked on it slowly but surely. They need new contributors though.

Thoreandan 4y ago

So fully-functional blob-free RPi3 / RPi4 is a thing now?

pabs3 4y ago
Not yet, but it could be if more people help contribute to it.

mmastrac 4y ago

I really hope they can get HDMI up on this because I'm struggling with an incompatibility between my IP KVM and Raspberry Pis. I would really love to dig into the HDMI initialization code. Something about the HDMI output on a Pi is just absolutely incompatible with the IP KVM's interface modules and it's maddeningly subtle because the Pis work _everywhere_ else.

grokx 4y ago

They should have called it "LiberPi"! :)

lproven 4y ago

Nobody's mentioned it yet, so... One reason this is so difficult is that the VPU runs an entire standalone OS of its own. AIUI, the VPU is in control of the computer, and the Arm is an accessory device. Only once the VPU has loaded its OS and it's running will the Arm be initialised, the kernel image loaded into RAM by the VPU, and then the Arm pointed at it. The OS continues to run and control the computer even once the Arm is running. The VPU runs a proprietary RTOS called ThreadX, which is now owned by Microsoft. https://en.wikipedia.org/wiki/ThreadX https://en.wikipedia.org/wiki/ThreadX

cleverca22 4y ago
the rpi-open-firmware codebase is going the route of doing as little as possible on the VPU, and giving the arm control fairly early i found that a bit limiting, with the lack of threads/drivers/modularity thats why i ported little-kernel, to get the same level of features threadx likely offered, and then be able to expand the feature-set far more with the lk-overlay codebase, you can drive the 2d and 3d cores from the VPU you can output ntsc or vga video you can turn on the arm core and boot linux and you can map /dev/fb0 on the video output and there are various cpu-independent features in LK like tga image decoding