6 ms·

The Raspberry Pi is really a VideoCore IV processor with an ARM bolted on the side. At power on, a boot ROM loads an embedded OS image into the VC4 processor. T

The Raspberry Pi is really a VideoCore IV processor with an ARM bolted on the side. At power on, a boot ROM loads an embedded OS image into the VC4 processor. That then sets up the hardware, powers up the ARM, and loads the OS image into that. The VC4 then continues running in the background servicing RPCs from the ARM. What this is is a replacement VC4 operating system image which just fires up the ARM with an embedded image, sets it running, and then halts (assuming I'm understanding the docs correctly); so it can't (yet) actually load an OS into the ARM from disk or respond to RPCs etc. In practical terms, this is the biggest, most difficult step forward towards running an almost completely open system on the RPi. (It could also be used as the core of a proper operating system running on the VC4 itself, which could be rather interesting.)

wyldfire 10y ago
> The Raspberry Pi is really a VideoCore IV processor with an ARM bolted on the side. I knew that the bootloader worked that way but didn't think of it being that way until reading your comment. Interesting to think of the hierarchy that way. Naively this makes me think that the ARM ISA could use something like UEFI (or BIOS) to unify the devices' bootstrapping logic around some common metaphor.
- tremon 10y ago
  Linux (and probably *BSD, not sure if they support it) can use devicetree files to configure itself for the hardware it's running on. No runtime firmware necessary.
- david-given 10y ago
  Well, the ARM is literally powered down until the VC4 turns it on. (I've heard that the only reason the ARM is there at all is that Broadcom had a royalty-free license and there was some spare silicon, so they thought, hey, why not, it might be useful...) Traditionally ARMs have never used any kind of common boot process because they come out of the embedded world where every system is bespoke --- the focus is always on individual products, rather than building a platform. There have been various efforts to fix this but AFAIAA they've never come to much. I think the current one is DeviceTree, but I don't believe there's much vendor support.
  - geon 10y ago
    > there was some spare silicon I don't think such a thing exists. Aren't dies made as small as possible to increase yield? I know the Cell processor in the ps3 had a spare spe core to significantly increase yield.
    - bronson 10y ago
      Chip layout/floorplan isn't like laying tiles. Not everything is gridded and fits together perfectly.
  - geon 10y ago
    > there was some spare silicon I don't think such a thing exists. Aren't dies made as small as possible to increase yield? I know the Cell processor in the ps3 had a spare spe core to significantly increase yield.
  - pjc50 10y ago
    The VC4 was intended for use in set-top-boxes or TVs, where the ARM would run a regular lightweight OS to drive the UI menus and the VC4 would do the video decoding, and compose the UI onto the video. It's not an accident, but it is a second-class citizen. It was also clearly intended for possible use in mobile phones, as some of the patent diagrams have "baseband" on them: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV---BCM2835-Overview https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor...
  - yalogin 10y ago
    > Traditionally ARMs have never used any kind of common boot process because they come out of the embedded world where every system is bespoke --- the focus is always on individual products, rather than building a platform. This is something I did not know. In the mobile world the boot happens in the UEFI mould right? So when someone licenses from ARM they get to design their own boot sequence?
    - pjc50 10y ago
      Mobile doesn't have UEFI either. Usually what happens is the processor has some minimal boot ROM that can initialise the Flash (and enforce signed boot!), and then it just loads a chunk of Flash into RAM and jumps to it.
- jdub 10y ago
  The ARM world is rapidly moving towards UEFI, partly thanks to Microsoft insisting there be something like it for them to build on. Even better, u-boot implements enough of it to boot a UEFI operating system. So on some new devices you'll get UEFI firmware, on everything else you can flash u-boot with UEFI support... and it's now part of their default configs. Pretty cool. :-)
  - c3833174 10y ago
    >The ARM world is rapidly moving towards UEFI, partly thanks to Microsoft insisting there be something like it for them to build on. Hope it isn't another quirky "just enough to boot windows, with other OSs having to clone windows behaviour to work" thing. Can secure boot be disabled on ARM?
    - wolfgke 10y ago
      > Can secure boot be disabled on ARM? The property whether it can be disabled depends on the concrete UEFI implementation.
    - pjc50 10y ago
      See your mobile phone: it depends whether the manufacturer makes that an option. In a processor I'm familiar with (imx53), once it's turned on by a processor fuse it stays on - it's then up to the second stage boot loader (provided by the device OEM) to decide whether to load untrusted images.
yalogin 10y ago
I read through the wiki page for videocore but still did not get a complete picture of what this is. Is it a GPU? If its a GPU or something that specializes in processing video/audio why is it in charge of the boot process? Doesn't it usually happen the other way around, the boot ROM loads the host OS which in turn initializes the other firmware?
- RossBencina 10y ago
  There is some information here: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV---BCM2835-Overview https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... In particular we are talking about "Scalar/Vector (VPU)" "Dual Core VideoCore IV® Multimedia Co-Processor" (Figure 3B). As far as I understand, that's the thing that boots the device and usually runs ThreadX OS in the closed-source bootloader. The VPU is a dual core processor with scalar and integer vector instructions. The VPU is more-or-less independent of the GPU, although I'm not sure whether the VPU gets involved with some GPU scheduling tasks. The GPU has four "QPU" QuadProcessor pipelines, which from memory can do floating-point vector processing.
  - Narishma 10y ago
    I believe the VPU runs the OpenGL ES stack when using the closed source drivers.
- pjc50 10y ago
  Better documentation: https://github.com/hermanhermitage/videocoreiv https://github.com/hermanhermitage/videocoreiv It's a vector processor and a GPU. The vector processor is intended for video decoding and therefore has a very big SIMD capability. It has some nice gimmicks like the 'square' register file: you can access rows or columns for SIMD operations. The VPU is I believe what boots first and normally runs an RTOS called "ThreadX". The GPU is OpenGL-capable.
makomk 10y ago
In practical terms, this seems to be quite a long way from running a completely open system - the VC4 is responsible for initializing a whole bunch of critical and I believe undocumented hardware that's required to enable any of the peripherals, and this doesn't. Basically, a whole bunch of stuff that'd be implemented by drivers on (say) most Allwinner SoCs I've looked at instead relies on the blob. This is probably part of the reason the Raspberry Pi has relatively good mainline kernel support - they don't have to deal with months of LKML hell to get basic things like clocking and power management working every time they release a new chip because that's all handled in the blob.
- david-given 10y ago
  I've written some bare-metal stuff on the Pi (somewhere I've got a 90% finished port of Fuzix for it), and oh god yes, that's so true. Being able to do complex stuff like set up a framebuffer by making a simple RPC call to the VC4, passing in a pointer and a descriptor block saying 'this big, please', is so much better than having to do it yourself. But this firmware doesn't contain code to initialise the GPU, it does contain initialisation code for one rather important peripheral, which is the DRAM. The VC4 has been pretty well reverse engineered for a while (I did one of the early C compiler ports for it), but the big missing piece was figuring out how to initialise the RAM. Without that, everything had to fit in the 128kB of built-in SRAM. This firmware's a huge step forward.
pcwalton 10y ago
Everyone says this setup where the VC4 boots the ARM is weird, but isn't that pretty similar to how Intel ME boots the x86, right down to the proprietary firmware and everything? It doesn't seem like an unusual setup to me at all: the only unusual thing, I guess, is that the VC4 is capable of a lot more than the small ARC/ARM core inside Intel ME.
- yellowapple 10y ago
  The main weirdness is that the VC4 is more of a GPU than a secondary CPU. It'd be like an Nvidia graphics card bootstrapping the actual CPU in an ordinary desktop.
  - pcwalton 10y ago
    Agreed that this is odd. But that's just a difference in the capabilities of the bootstrap core: Intel uses a less powerful ARC or ARM core, while Broadcom uses a more powerful core that also functions as a GPU. The mechanism is the same. There isn't a fundamental difference in how Intel CPUs start up and how the Raspberry Pi starts up, as I sometimes see people implying.
  - david-given 10y ago
    That's not quite right. The Pi has a completely separate GPU unit. The VC4 is different, and is a pretty normal processor, although it's got a pile of DSP addons. The instruction set's here: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-Programmers-Manual https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... ...but to summarise: - 32 registers which can hold either integers or 32-bit floats; - ARM-like 32-bit 3op instructions, with a limited set of 16-bit 2op instructions; - integrated 32-bit floating point instructions, using the same registers as everything else; - some 48 bit instruction forms (allowing a full 32-bit payload! No ghastly ARM constant pools or PowerPC-style split payloads!); - two cores, with integrated interrupt handler; - integrated DSP with 80-bit vector instructions working on a 64x64x8bit vector working area, which TBH looks like it was stolen from another processor completely and which I frankly don't understand; - ARM-style conditional execution (for some instructions); - ARM-style multiregister loads and saves (and pushes and pops); - DSP-style saturated arithmetic and fixed-point support; - no ALU-setting-condition-flags instructions, or add-with-carry or subtract-with-carry operations, which makes 64-bit arithmetic really painful (if you need cmp to set the condition flags, what's the carry flag even for?) It's actually a really nice thing to write assembly for. It's orthogonal enough to be understandable, while quirky enough to allow some really satisfying optimisations. e.g. there's the addcmpb instruction, which while add a value to a register, compare with another register, and branch based on a condition code, all in a single 32-bit operation --- it's basically a loop in a box. It's also the only time I've ever seen 6-bit floating point constants...

wyldfire 10y ago

> The Raspberry Pi is really a VideoCore IV processor with an ARM bolted on the side. I knew that the bootloader worked that way but didn't think of it being that way until reading your comment. Interesting to think of the hierarchy that way. Naively this makes me think that the ARM ISA could use something like UEFI (or BIOS) to unify the devices' bootstrapping logic around some common metaphor.

tremon 10y ago
Linux (and probably *BSD, not sure if they support it) can use devicetree files to configure itself for the hardware it's running on. No runtime firmware necessary.
david-given 10y ago
Well, the ARM is literally powered down until the VC4 turns it on. (I've heard that the only reason the ARM is there at all is that Broadcom had a royalty-free license and there was some spare silicon, so they thought, hey, why not, it might be useful...) Traditionally ARMs have never used any kind of common boot process because they come out of the embedded world where every system is bespoke --- the focus is always on individual products, rather than building a platform. There have been various efforts to fix this but AFAIAA they've never come to much. I think the current one is DeviceTree, but I don't believe there's much vendor support.
- geon 10y ago
  > there was some spare silicon I don't think such a thing exists. Aren't dies made as small as possible to increase yield? I know the Cell processor in the ps3 had a spare spe core to significantly increase yield.
  - bronson 10y ago
    Chip layout/floorplan isn't like laying tiles. Not everything is gridded and fits together perfectly.
- geon 10y ago
  > there was some spare silicon I don't think such a thing exists. Aren't dies made as small as possible to increase yield? I know the Cell processor in the ps3 had a spare spe core to significantly increase yield.
- pjc50 10y ago
  The VC4 was intended for use in set-top-boxes or TVs, where the ARM would run a regular lightweight OS to drive the UI menus and the VC4 would do the video decoding, and compose the UI onto the video. It's not an accident, but it is a second-class citizen. It was also clearly intended for possible use in mobile phones, as some of the patent diagrams have "baseband" on them: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV---BCM2835-Overview https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor...
- yalogin 10y ago
  > Traditionally ARMs have never used any kind of common boot process because they come out of the embedded world where every system is bespoke --- the focus is always on individual products, rather than building a platform. This is something I did not know. In the mobile world the boot happens in the UEFI mould right? So when someone licenses from ARM they get to design their own boot sequence?
  - pjc50 10y ago
    Mobile doesn't have UEFI either. Usually what happens is the processor has some minimal boot ROM that can initialise the Flash (and enforce signed boot!), and then it just loads a chunk of Flash into RAM and jumps to it.
jdub 10y ago
The ARM world is rapidly moving towards UEFI, partly thanks to Microsoft insisting there be something like it for them to build on. Even better, u-boot implements enough of it to boot a UEFI operating system. So on some new devices you'll get UEFI firmware, on everything else you can flash u-boot with UEFI support... and it's now part of their default configs. Pretty cool. :-)
- c3833174 10y ago
  >The ARM world is rapidly moving towards UEFI, partly thanks to Microsoft insisting there be something like it for them to build on. Hope it isn't another quirky "just enough to boot windows, with other OSs having to clone windows behaviour to work" thing. Can secure boot be disabled on ARM?
  - wolfgke 10y ago
    > Can secure boot be disabled on ARM? The property whether it can be disabled depends on the concrete UEFI implementation.
  - pjc50 10y ago
    See your mobile phone: it depends whether the manufacturer makes that an option. In a processor I'm familiar with (imx53), once it's turned on by a processor fuse it stays on - it's then up to the second stage boot loader (provided by the device OEM) to decide whether to load untrusted images.

yalogin 10y ago

I read through the wiki page for videocore but still did not get a complete picture of what this is. Is it a GPU? If its a GPU or something that specializes in processing video/audio why is it in charge of the boot process? Doesn't it usually happen the other way around, the boot ROM loads the host OS which in turn initializes the other firmware?

RossBencina 10y ago
There is some information here: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV---BCM2835-Overview https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... In particular we are talking about "Scalar/Vector (VPU)" "Dual Core VideoCore IV® Multimedia Co-Processor" (Figure 3B). As far as I understand, that's the thing that boots the device and usually runs ThreadX OS in the closed-source bootloader. The VPU is a dual core processor with scalar and integer vector instructions. The VPU is more-or-less independent of the GPU, although I'm not sure whether the VPU gets involved with some GPU scheduling tasks. The GPU has four "QPU" QuadProcessor pipelines, which from memory can do floating-point vector processing.
- Narishma 10y ago
  I believe the VPU runs the OpenGL ES stack when using the closed source drivers.
pjc50 10y ago
Better documentation: https://github.com/hermanhermitage/videocoreiv https://github.com/hermanhermitage/videocoreiv It's a vector processor and a GPU. The vector processor is intended for video decoding and therefore has a very big SIMD capability. It has some nice gimmicks like the 'square' register file: you can access rows or columns for SIMD operations. The VPU is I believe what boots first and normally runs an RTOS called "ThreadX". The GPU is OpenGL-capable.

makomk 10y ago

In practical terms, this seems to be quite a long way from running a completely open system - the VC4 is responsible for initializing a whole bunch of critical and I believe undocumented hardware that's required to enable any of the peripherals, and this doesn't. Basically, a whole bunch of stuff that'd be implemented by drivers on (say) most Allwinner SoCs I've looked at instead relies on the blob. This is probably part of the reason the Raspberry Pi has relatively good mainline kernel support - they don't have to deal with months of LKML hell to get basic things like clocking and power management working every time they release a new chip because that's all handled in the blob.

david-given 10y ago
I've written some bare-metal stuff on the Pi (somewhere I've got a 90% finished port of Fuzix for it), and oh god yes, that's so true. Being able to do complex stuff like set up a framebuffer by making a simple RPC call to the VC4, passing in a pointer and a descriptor block saying 'this big, please', is so much better than having to do it yourself. But this firmware doesn't contain code to initialise the GPU, it does contain initialisation code for one rather important peripheral, which is the DRAM. The VC4 has been pretty well reverse engineered for a while (I did one of the early C compiler ports for it), but the big missing piece was figuring out how to initialise the RAM. Without that, everything had to fit in the 128kB of built-in SRAM. This firmware's a huge step forward.

pcwalton 10y ago

Everyone says this setup where the VC4 boots the ARM is weird, but isn't that pretty similar to how Intel ME boots the x86, right down to the proprietary firmware and everything? It doesn't seem like an unusual setup to me at all: the only unusual thing, I guess, is that the VC4 is capable of a lot more than the small ARC/ARM core inside Intel ME.

yellowapple 10y ago
The main weirdness is that the VC4 is more of a GPU than a secondary CPU. It'd be like an Nvidia graphics card bootstrapping the actual CPU in an ordinary desktop.
- pcwalton 10y ago
  Agreed that this is odd. But that's just a difference in the capabilities of the bootstrap core: Intel uses a less powerful ARC or ARM core, while Broadcom uses a more powerful core that also functions as a GPU. The mechanism is the same. There isn't a fundamental difference in how Intel CPUs start up and how the Raspberry Pi starts up, as I sometimes see people implying.
- david-given 10y ago
  That's not quite right. The Pi has a completely separate GPU unit. The VC4 is different, and is a pretty normal processor, although it's got a pile of DSP addons. The instruction set's here: https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV-Programmers-Manual https://github.com/hermanhermitage/videocoreiv/wiki/VideoCor... ...but to summarise: - 32 registers which can hold either integers or 32-bit floats; - ARM-like 32-bit 3op instructions, with a limited set of 16-bit 2op instructions; - integrated 32-bit floating point instructions, using the same registers as everything else; - some 48 bit instruction forms (allowing a full 32-bit payload! No ghastly ARM constant pools or PowerPC-style split payloads!); - two cores, with integrated interrupt handler; - integrated DSP with 80-bit vector instructions working on a 64x64x8bit vector working area, which TBH looks like it was stolen from another processor completely and which I frankly don't understand; - ARM-style conditional execution (for some instructions); - ARM-style multiregister loads and saves (and pushes and pops); - DSP-style saturated arithmetic and fixed-point support; - no ALU-setting-condition-flags instructions, or add-with-carry or subtract-with-carry operations, which makes 64-bit arithmetic really painful (if you need cmp to set the condition flags, what's the carry flag even for?) It's actually a really nice thing to write assembly for. It's orthogonal enough to be understandable, while quirky enough to allow some really satisfying optimisations. e.g. there's the addcmpb instruction, which while add a value to a register, compare with another register, and branch based on a condition code, all in a single 32-bit operation --- it's basically a loop in a box. It's also the only time I've ever seen 6-bit floating point constants...