4 ms·

Whew, kernel bugs due to complexities of live patching code! (https://nickdesaulniers.github.io/blog/2020/04/06/off-by-two/ https://nickdesaulniers.github.io/bl

Whew, kernel bugs due to complexities of live patching code! (https://nickdesaulniers.github.io/blog/2020/04/06/off-by-two/ https://nickdesaulniers.github.io/blog/2020/04/06/off-by-two...) high fives...anyone...anyone...?

stefan_ 6y ago
Nick, just dropping by to say that I have enjoyed your blog posts on all topics kernel development tremendously. Thanks!
wyldfire 6y ago
I enjoyed this when it was posted previously. Keep up the good work, Nick.
marcan_42 6y ago
Oh my, that last bit from Linus there. Been there done that, on a modern system! So I was making Linux run on a PlayStation3 booting from an exploit, so that we could have full access to the hardware/GPU (not the locked down PS3 Linux support that it came with, and had since been retroactively removed in an update). This is running on top of a built-in hypervisor, on a PowerPC CPU. I had to debug early bring-up code of the Linux bootloader I was writing from scratch (AsbestOS), and at that point you don't have any real hardware access. There is an internal serial port but it's not accessible to VMs, graphics is way too hard to bring up, USB is also a pain. No usable LEDs too. So what is there? Panic. The hypervisor panic hypercall had an argument with two modes: you could either reboot, or shut down with a beep. So that was my boolean output primitive, which at one point I used to "print" out an address, bit by bit, during debugging the assembler bringup code. https://github.com/marcan/asbestos/blob/master/stage2/start.S#L81 https://github.com/marcan/asbestos/blob/master/stage2/start.... Once I got into C and I could start writing more interesting code, I eventually graduated from panic/reboot to using Ethernet. The Ethernet device was virtualized to some extent. You still had to write DMA descriptors to get packets out, but you didn't have to worry about low-level hardware init. So I stuffed some hand-crafted broadcast UDP packets in there, and the descriptors to blast them out, and called the "start transmit" Ethernet device hypercalls to get printf-over-UDP working. Fun thing is the PS3 has a built-in Ethernet switch with VLANs, so you needed to stuff VLAN tags into the packets if you wanted them to make it out of the Ethernet port. Until the PS3 slim, when they got rid of that switch, and then you no longer needed tags. Fun. https://github.com/marcan/asbestos/blob/master/stage2/debug.c https://github.com/marcan/asbestos/blob/master/stage2/debug.... I used this same UDP-over-LV1-hv-ethernet debugging tool to debug early kernel startup too, after the bootloader hands off, since we had to revamp chunks of the Linux on PS3 early memory startup code (as the memory config while booting in "Game" mode was very different from the formerly officially supported "Other OS" mode) way before graphics was up. This code was eventually upstreamed: https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/ps3/gelic_udbg.c https://github.com/torvalds/linux/blob/master/arch/powerpc/p... By comparison, when I was doing (unofficial) Linux on PS4 we had the luxury of running on bare metal, and there were testpoints for a serial port on the motherboard, so we could just poke data out of there with some trivial UART code, and use the existing Linux earlycon support for the same after handoff (though the baud rate multiplier was wrong... and also, bizarrely enough, it's an "XScale" ARM variant 8250 port because the PS4's southbridge is a repurposed Marvell ARM SoC with a PCIe bridge to the x86!).
- ndesaulniers 6y ago
  Wow! That's a great read, you should post a copy to your blog. I remember running yellow dog Linux on my PS3 (as a kid) but didn't know the GPU was inaccessible. For the hypervisor, did you have to do any kind of RAM training? Are folks running Linux on PS4's these days? I haven't been keeping up to much with the system exploit scene (had exploited Xbox 360, PSP) other than the excellent YouTube channel "modern vintage gamer."

stefan_ 6y ago

Nick, just dropping by to say that I have enjoyed your blog posts on all topics kernel development tremendously. Thanks!

wyldfire 6y ago

I enjoyed this when it was posted previously. Keep up the good work, Nick.

marcan_42 6y ago

Oh my, that last bit from Linus there. Been there done that, on a modern system! So I was making Linux run on a PlayStation3 booting from an exploit, so that we could have full access to the hardware/GPU (not the locked down PS3 Linux support that it came with, and had since been retroactively removed in an update). This is running on top of a built-in hypervisor, on a PowerPC CPU. I had to debug early bring-up code of the Linux bootloader I was writing from scratch (AsbestOS), and at that point you don't have any real hardware access. There is an internal serial port but it's not accessible to VMs, graphics is way too hard to bring up, USB is also a pain. No usable LEDs too. So what is there? Panic. The hypervisor panic hypercall had an argument with two modes: you could either reboot, or shut down with a beep. So that was my boolean output primitive, which at one point I used to "print" out an address, bit by bit, during debugging the assembler bringup code. https://github.com/marcan/asbestos/blob/master/stage2/start.S#L81 https://github.com/marcan/asbestos/blob/master/stage2/start.... Once I got into C and I could start writing more interesting code, I eventually graduated from panic/reboot to using Ethernet. The Ethernet device was virtualized to some extent. You still had to write DMA descriptors to get packets out, but you didn't have to worry about low-level hardware init. So I stuffed some hand-crafted broadcast UDP packets in there, and the descriptors to blast them out, and called the "start transmit" Ethernet device hypercalls to get printf-over-UDP working. Fun thing is the PS3 has a built-in Ethernet switch with VLANs, so you needed to stuff VLAN tags into the packets if you wanted them to make it out of the Ethernet port. Until the PS3 slim, when they got rid of that switch, and then you no longer needed tags. Fun. https://github.com/marcan/asbestos/blob/master/stage2/debug.c https://github.com/marcan/asbestos/blob/master/stage2/debug.... I used this same UDP-over-LV1-hv-ethernet debugging tool to debug early kernel startup too, after the bootloader hands off, since we had to revamp chunks of the Linux on PS3 early memory startup code (as the memory config while booting in "Game" mode was very different from the formerly officially supported "Other OS" mode) way before graphics was up. This code was eventually upstreamed: https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/ps3/gelic_udbg.c https://github.com/torvalds/linux/blob/master/arch/powerpc/p... By comparison, when I was doing (unofficial) Linux on PS4 we had the luxury of running on bare metal, and there were testpoints for a serial port on the motherboard, so we could just poke data out of there with some trivial UART code, and use the existing Linux earlycon support for the same after handoff (though the baud rate multiplier was wrong... and also, bizarrely enough, it's an "XScale" ARM variant 8250 port because the PS4's southbridge is a repurposed Marvell ARM SoC with a PCIe bridge to the x86!).

ndesaulniers 6y ago
Wow! That's a great read, you should post a copy to your blog. I remember running yellow dog Linux on my PS3 (as a kid) but didn't know the GPU was inaccessible. For the hypervisor, did you have to do any kind of RAM training? Are folks running Linux on PS4's these days? I haven't been keeping up to much with the system exploit scene (had exploited Xbox 360, PSP) other than the excellent YouTube channel "modern vintage gamer."