5 ms·

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itanium

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itaniums in the past. VideoBIOS expects to run and expects a well behaving Intel CPU to do the power-up. That said X can sometimes emulate these quite well. On ARM we’d also run into alignment issues and likely other quirks - but in principle...

jchw 7y ago
I always wondered what the deal was with Video BIOS on modern PCI-express graphics cards. Obviously it must exist, since int 10h still works. Does video BIOS even need to be executed, though? I always assumed it was only necessary for the primary card to be able to display output during boot on PCs. (Otherwise, wouldn't two different cards trample over eachother's implementation of int 10h?) Sorry for the probably-obvious questions, it's sometimes tricky to find good sources for weird information like this that are also up to date.
- qdot_me 7y ago
  It’s all, cough, implementation dependent. You can run two VideoBIOS implementations, and chances are only one will install an int10h handler, but both will set up the registers and whatnot in the GPUs. My AMD TAHITIs for instance need VideoBIOS to start some form of thermal management loop - otherwise they just run full-throttle on the fan. Then whichever card prevails (BIOS has the ability to select the initialization order) becomes the boot display device. X11 has some (generally working, for well behaved GPUs) emulation of this environment, so that the GPU can initialize late, and even reset under X control. This is how sane cards can work under headless ARM etc. Now, some manufacturers assume you get something like SSE or MMX - VideoBIOS spec technically mandates 386 instruction set only. That crap gets badly emulated. On top of this, drivers can sometimes reinit anyways, from native kernel code. If that happens, the VideoBIOS concerns are moot.
  - jchw 7y ago
    Thanks for the information. I’ve been really curious about this with the incoming ARM dev boards that have full PCI-e edge connectors. Pretty fascinating that X11 of all things is dealing with Video BIOS initialization, though. The sheer amount of functionality that was overloaded into X...
    - floatboth 7y ago
      You don't need X, and I wouldn't assume its remnants of userspace mode setting would work on non-x86. The amdgpu kernel driver POSTs the GPUs when loaded just fine! I run an RX 480 on a MACCHIATObin with FreeBSD and Wayland only :) Plus, there are two ways to get the GPU in UEFI already – X86EmulatorPkg and native aarch64 GOP driver builds provided by AMD.
      - jchw 7y ago
        X86EmulatorPkg is pretty cool. I guess if you have a board with UEFI it only makes sense to use something like that.
  - sebazzz 7y ago
    I always assumed the VideoBIOS runs on the GPU, but if I understand you correctly this is code run on the CPU?
    - floatboth 7y ago
      Yes — "VBIOS", and these days mostly "UEFI GOP driver", is code that runs on the CPU and allows the system firmware to initialize the card and preset video output before any operating system is booted. Without it, you cannot enter the setup screen graphically, only over a serial port :D
    - ComputerGuru 7y ago
      Yes, the PC (or whatever) hands off control of the entire system to the VideoBIOS which must do its thing and then pass control back.
pm215 7y ago
People have got at least some cards working on server-class arm setups. On the other hand I think they might be using the 'emulate x86 uefi rom in an arm uefi system' hack, so there might be more effort required on a non-uefi setup. Still, it should have shaken out most of the alignment and memory-cacheability-attibutes issues already, hopefully.
- floatboth 7y ago
  Yep, the "hack" (X86EmulatorPkg) allows running the card in the UEFI out of the box. AMD also provides native aarch64 builds of their UEFI GOP driver though. And none of this is necessary for running the GPU in the OS – amdgpu POSTs the GPU just fine. > memory-cacheability-attibutes issues Recently I've added aarch64 support to FreeBSD's port of the DRM/KMS drivers :) Took a couple hours to realize that our implementations of Linux's mapping functions used normal uncacheable memory instead of device memory – fixing that stopped the hangs on driver load and allowed everything to work. Then there was some corruption on the screen – our drm-kms is from Linux 5.0 for now, and I've had to cherry-pick a fix that only landed in 5.1 I think: https://patchwork.kernel.org/patch/10778815/ https://patchwork.kernel.org/patch/10778815/
  - pm215 7y ago
    You wouldn't believe how much discussion it took to decide that that minor drm_cache.h change really was the right fix for that issue :-)
timthorn 7y ago
Some AMD cards have an Arm build of their UEFI firmware available, which does allow video from boot: https://developer.arm.com/architectures/system-architectures/software-standards/uefi-drivers https://developer.arm.com/architectures/system-architectures...
floatboth 7y ago
VBIOS is often not necessary for running a GPU in the OS. The amdgpu driver can POST a GPU by itself just fine. Still… X86EmulatorPkg allows running an amd64 VBIOS in UEFI on an aarch64 machine :) AFAIK the bigger problem on embedded boards is half assed Synopsys Designware host controllers. I have a Radeon running on my Marvell MACCHIATObin, on FreeBSD even. But from what I've heard the Rockchip RK3399 has a worse version of the controller, and people trying GPUs on the ROCKPro64 saw errors related to not large enough BAR space or something. UPD: yeah, someone in the thread mentioned BAR space issues wrt NXP i.MX SoCs, that's probably what's happening on Rockchip. Would be amazing if the Broadcom chip in the Pi turns out to be the one with enough BAR space! :D
- markus92 7y ago
  Could you clarify to me/give a short definition of BAR space? For obvious reasons it's a bit hard to search for :)
  - cesarb 7y ago
    In PCI, BAR is Base Address Register, which is a register in the PCI device's configuration space which defines where in the machine's physical memory address space that particular window of memory and/or I/O will be mapped (a single device can have several BARs, for instance a simple graphics card could have one for its control registers and one for the framebuffer). So the "BAR space" would be a shorthand for "the region of the physical memory address space which can be used to map the PCI devices memory through their Base Address Registers". The size of this region is limited, and graphics cards in particular tend to have somewhat large BARs. (See for yourself in your machine: run "lspci -v", the lines starting with "Memory at ..." or "I/O ports at ..." are the BARs.)
  - bluecmd 7y ago
    Search "PCIe bar space" and you should be fine
Teknoman117 7y ago
SiFive has a few videos where they have their RISC-V core running an AMD gpu (via the in kernel amdgpu driver).
bstar77 7y ago
"GPUs are notoriously hard to get to work on non-intel" I don't understand this statement... accelerated graphics have no issues working on any x86 system as well as ARM based linux systems (not to mention every cell phone in the past 10 years). Itanium is also Intel so that statement contradicts your previous statement.
- qdot_me 7y ago
  To a limited extent. The accelerated graphics on ARM (and cell phones) has always been a system integrator problem. Until very recently, they were not user customizable - which means the initialisation was part of the proprietary firmware and hard to port across even different implementations of the same SoC. The x86 world had the advantage of user upgradable GPUs, which necessitates standardization and common firmware. On top of which, the accelerated graphics of cell phones is a horrible kludge of various standards.
notaplumber 7y ago
OpenBSD has radeondrm enabled on mips64, powerpc and arm64. It can initialize the board just fine without emulating x86, since KMS. Working 3D acceleration is another story. OpenBSD even has (or had, build breaks often) packages for chromium browser on arm64 that were tested on this.
- floatboth 7y ago
  > Working 3D acceleration is another story Why? I recently enabled amdgpu on FreeBSD arm64 — as soon as KMS happened, 3D just worked. There was some buffer corruption (https://user-images.githubusercontent.com/208340/60443774-97243900-9c24-11e9-861a-6dd29101c235.png https://user-images.githubusercontent.com/208340/60443774-97...) for which I've had to cherry-pick this tiny Linux patch: https://patchwork.kernel.org/patch/10778815/ https://patchwork.kernel.org/patch/10778815/ > packages for chromium browser on arm64 And we have Firefox, which doesn't break, has a working JS JIT, works on Wayland, renders with WebRender :)
vardump 7y ago
> On ARM we’d also run into alignment issues... Just don't use ARMv5 or older. ARMv6+ supports unaligned access.
- planteen 7y ago
  At least on ARMv7, it's up to the MMU, not the CPU. I've worked with Cortex-A9s that trap the unaligned access. You can write a handler or crash.
  - vardump 7y ago
    Interesting. I've kept my accesses aligned, but noticed that unaligned ones worked. Didn't know about the exception mechanism behind, but yeah, should have err... expected it. By the way, you can also opt to crash on x86. Just need to enable bit AM (Alignment Mask) bit 18) in CR0 (kernel stuff) and AC (Alignment Check, also bit 18) in EFLAGS. After that point, unaligned accesses trap. I use this in my CPU JIT/dynamic compiler to trap target unaligned accesses. https://en.wikipedia.org/wiki/Control_register#CR0 https://en.wikipedia.org/wiki/Control_register#CR0

jchw 7y ago

I always wondered what the deal was with Video BIOS on modern PCI-express graphics cards. Obviously it must exist, since int 10h still works. Does video BIOS even need to be executed, though? I always assumed it was only necessary for the primary card to be able to display output during boot on PCs. (Otherwise, wouldn't two different cards trample over eachother's implementation of int 10h?) Sorry for the probably-obvious questions, it's sometimes tricky to find good sources for weird information like this that are also up to date.

qdot_me 7y ago
It’s all, cough, implementation dependent. You can run two VideoBIOS implementations, and chances are only one will install an int10h handler, but both will set up the registers and whatnot in the GPUs. My AMD TAHITIs for instance need VideoBIOS to start some form of thermal management loop - otherwise they just run full-throttle on the fan. Then whichever card prevails (BIOS has the ability to select the initialization order) becomes the boot display device. X11 has some (generally working, for well behaved GPUs) emulation of this environment, so that the GPU can initialize late, and even reset under X control. This is how sane cards can work under headless ARM etc. Now, some manufacturers assume you get something like SSE or MMX - VideoBIOS spec technically mandates 386 instruction set only. That crap gets badly emulated. On top of this, drivers can sometimes reinit anyways, from native kernel code. If that happens, the VideoBIOS concerns are moot.
- jchw 7y ago
  Thanks for the information. I’ve been really curious about this with the incoming ARM dev boards that have full PCI-e edge connectors. Pretty fascinating that X11 of all things is dealing with Video BIOS initialization, though. The sheer amount of functionality that was overloaded into X...
  - floatboth 7y ago
    You don't need X, and I wouldn't assume its remnants of userspace mode setting would work on non-x86. The amdgpu kernel driver POSTs the GPUs when loaded just fine! I run an RX 480 on a MACCHIATObin with FreeBSD and Wayland only :) Plus, there are two ways to get the GPU in UEFI already – X86EmulatorPkg and native aarch64 GOP driver builds provided by AMD.
    - jchw 7y ago
      X86EmulatorPkg is pretty cool. I guess if you have a board with UEFI it only makes sense to use something like that.
- sebazzz 7y ago
  I always assumed the VideoBIOS runs on the GPU, but if I understand you correctly this is code run on the CPU?
  - floatboth 7y ago
    Yes — "VBIOS", and these days mostly "UEFI GOP driver", is code that runs on the CPU and allows the system firmware to initialize the card and preset video output before any operating system is booted. Without it, you cannot enter the setup screen graphically, only over a serial port :D
  - ComputerGuru 7y ago
    Yes, the PC (or whatever) hands off control of the entire system to the VideoBIOS which must do its thing and then pass control back.

pm215 7y ago

People have got at least some cards working on server-class arm setups. On the other hand I think they might be using the 'emulate x86 uefi rom in an arm uefi system' hack, so there might be more effort required on a non-uefi setup. Still, it should have shaken out most of the alignment and memory-cacheability-attibutes issues already, hopefully.

floatboth 7y ago
Yep, the "hack" (X86EmulatorPkg) allows running the card in the UEFI out of the box. AMD also provides native aarch64 builds of their UEFI GOP driver though. And none of this is necessary for running the GPU in the OS – amdgpu POSTs the GPU just fine. > memory-cacheability-attibutes issues Recently I've added aarch64 support to FreeBSD's port of the DRM/KMS drivers :) Took a couple hours to realize that our implementations of Linux's mapping functions used normal uncacheable memory instead of device memory – fixing that stopped the hangs on driver load and allowed everything to work. Then there was some corruption on the screen – our drm-kms is from Linux 5.0 for now, and I've had to cherry-pick a fix that only landed in 5.1 I think: https://patchwork.kernel.org/patch/10778815/ https://patchwork.kernel.org/patch/10778815/
- pm215 7y ago
  You wouldn't believe how much discussion it took to decide that that minor drm_cache.h change really was the right fix for that issue :-)

timthorn 7y ago

Some AMD cards have an Arm build of their UEFI firmware available, which does allow video from boot: https://developer.arm.com/architectures/system-architectures/software-standards/uefi-drivers https://developer.arm.com/architectures/system-architectures...

floatboth 7y ago

VBIOS is often not necessary for running a GPU in the OS. The amdgpu driver can POST a GPU by itself just fine. Still… X86EmulatorPkg allows running an amd64 VBIOS in UEFI on an aarch64 machine :) AFAIK the bigger problem on embedded boards is half assed Synopsys Designware host controllers. I have a Radeon running on my Marvell MACCHIATObin, on FreeBSD even. But from what I've heard the Rockchip RK3399 has a worse version of the controller, and people trying GPUs on the ROCKPro64 saw errors related to not large enough BAR space or something. UPD: yeah, someone in the thread mentioned BAR space issues wrt NXP i.MX SoCs, that's probably what's happening on Rockchip. Would be amazing if the Broadcom chip in the Pi turns out to be the one with enough BAR space! :D

markus92 7y ago
Could you clarify to me/give a short definition of BAR space? For obvious reasons it's a bit hard to search for :)
- cesarb 7y ago
  In PCI, BAR is Base Address Register, which is a register in the PCI device's configuration space which defines where in the machine's physical memory address space that particular window of memory and/or I/O will be mapped (a single device can have several BARs, for instance a simple graphics card could have one for its control registers and one for the framebuffer). So the "BAR space" would be a shorthand for "the region of the physical memory address space which can be used to map the PCI devices memory through their Base Address Registers". The size of this region is limited, and graphics cards in particular tend to have somewhat large BARs. (See for yourself in your machine: run "lspci -v", the lines starting with "Memory at ..." or "I/O ports at ..." are the BARs.)
- bluecmd 7y ago
  Search "PCIe bar space" and you should be fine

Teknoman117 7y ago

SiFive has a few videos where they have their RISC-V core running an AMD gpu (via the in kernel amdgpu driver).

bstar77 7y ago

"GPUs are notoriously hard to get to work on non-intel" I don't understand this statement... accelerated graphics have no issues working on any x86 system as well as ARM based linux systems (not to mention every cell phone in the past 10 years). Itanium is also Intel so that statement contradicts your previous statement.

qdot_me 7y ago
To a limited extent. The accelerated graphics on ARM (and cell phones) has always been a system integrator problem. Until very recently, they were not user customizable - which means the initialisation was part of the proprietary firmware and hard to port across even different implementations of the same SoC. The x86 world had the advantage of user upgradable GPUs, which necessitates standardization and common firmware. On top of which, the accelerated graphics of cell phones is a horrible kludge of various standards.

notaplumber 7y ago

OpenBSD has radeondrm enabled on mips64, powerpc and arm64. It can initialize the board just fine without emulating x86, since KMS. Working 3D acceleration is another story. OpenBSD even has (or had, build breaks often) packages for chromium browser on arm64 that were tested on this.

floatboth 7y ago
> Working 3D acceleration is another story Why? I recently enabled amdgpu on FreeBSD arm64 — as soon as KMS happened, 3D just worked. There was some buffer corruption (https://user-images.githubusercontent.com/208340/60443774-97243900-9c24-11e9-861a-6dd29101c235.png https://user-images.githubusercontent.com/208340/60443774-97...) for which I've had to cherry-pick this tiny Linux patch: https://patchwork.kernel.org/patch/10778815/ https://patchwork.kernel.org/patch/10778815/ > packages for chromium browser on arm64 And we have Firefox, which doesn't break, has a working JS JIT, works on Wayland, renders with WebRender :)

vardump 7y ago

> On ARM we’d also run into alignment issues... Just don't use ARMv5 or older. ARMv6+ supports unaligned access.

planteen 7y ago
At least on ARMv7, it's up to the MMU, not the CPU. I've worked with Cortex-A9s that trap the unaligned access. You can write a handler or crash.
- vardump 7y ago
  Interesting. I've kept my accesses aligned, but noticed that unaligned ones worked. Didn't know about the exception mechanism behind, but yeah, should have err... expected it. By the way, you can also opt to crash on x86. Just need to enable bit AM (Alignment Mask) bit 18) in CR0 (kernel stuff) and AC (Alignment Check, also bit 18) in EFLAGS. After that point, unaligned accesses trap. I use this in my CPU JIT/dynamic compiler to trap target unaligned accesses. https://en.wikipedia.org/wiki/Control_register#CR0 https://en.wikipedia.org/wiki/Control_register#CR0