4 ms·

I honestly love how more people are rolling their own OS's. But it stinks how difficult hardware support is. I wish there was a nice wrapper dedicated to abstr

I honestly love how more people are rolling their own OS's. But it stinks how difficult hardware support is. I wish there was a nice wrapper dedicated to abstracting hardware support better so developers can focus on the interesting parts instead of arguing with wi-fi cards.

cjdell 6y ago
Completely agree. Something like Project Treble, but for PC hardware. Thinking about it, this is what the BIOS was originally for: to provide a bare minimum abstraction to the hardware. I hoped that UEFI would extend this abstraction in a meaningful way, making OS development easier. Instead it has made it even more complex.
- hapless 6y ago
  The original EFI spec very much did extend that abstraction. You can do a lot in an EFI environment if the hardware plays ball: network, disk, video etc UEFI, sadly, added support for raw x86 option ROMs, to some extent casting us back into the "old" world.
userbinator 6y ago
In the earlier days of the x86 PC, the hardware was far more documented, but now a lot of manufacturers are notoriously secretive about it. Nonetheless, standards still exist (HDA,ATA/ATAPI/SATA,OHCI/UHCI/EHCI/XHCI,etc.) and much of the backwards compatibility is still there. You bring up wi-fi cards and that is definitely one category of hardware with little standardisation/documentation (I suspect due to FCC regulations and such.) Wired ethernet is a bit better --- Realtek and Intel NICs are both quite well-documented, with the latter being more complex. The other difficult one is GPU acceleration (but a VESA framebuffer is usually available, which gets you at least a GUI if not a fast one.)
Koshkin 6y ago
> a nice wrapper Are Linux device drivers not nice enough? Hell, why couldn’t one think, for that matter, of a minimal Linux kernel as some kind of BIOS. Incidentally, we already use it as a hypervisor, so you could probably even include support for the devices you want in the hypervisor itself.
- juancampa 6y ago
  That's the approach urbit has taken
- tenebrisalietum 6y ago
  There are projects that load Linux as a UEFI DXE, and coreboot pretty much can be configured to initialize the platform and then hand over to a Linux payload immediately.
skissane 6y ago
> I wish there was a nice wrapper dedicated to abstracting hardware support better so developers can focus on the interesting parts The University of Utah had a research project back in the late 1990s which offered exactly that – the OSKit project [0]. A set of libraries for common (at the time) x86 hardware so OS developers and researchers could focus on what was original about their OS instead of writing yet another hard disk or Ethernet driver Unfortunately, the research project ended circa 2000 and it is dead now. No support for contemporary devices like Wifi and USB. But, if you just want something to run in a VM (QEMU or VirtualBox or whatever) it probably still works. [0] https://www.cs.utah.edu/flux/oskit/ https://www.cs.utah.edu/flux/oskit/
- tooltower 6y ago
  Today, when I build a custom Linux kernel, and am playing with config vars to decide what is our isn't included in the build, I very much tend to think of it as assembling a kernel from various library pieces. How was OSkit different?
  - skissane 6y ago
    Linux isn't written with the use case of using bits of it in isolation in mind. Sure, you can do that, but the Linux developers don't have making that easy as a design goal, so it will often be quite hard, and even when it isn't so hard that is just an accident. By contrast, the OSKit developers had as an explicit design goal to make it easy to use bits of their code in isolation from the rest. Linux gives no API stability guarantees for its internal APIs, only for the system call interface. By contrast, the OSKit developers intended the API of each component to be a stable documented interface. The Linux developers don't want Linux's internal APIs to be stable documented interfaces because that has a cost in that it slows down refactoring, which in turn reduces the various benefits that refactoring can produce (better performance, better maintainability, new features, etc)
    - octorian 6y ago
      > The Linux developers don't want Linux's internal APIs to be stable documented interfaces It also has the "benefit" of making it extremely difficult to actually distribute stable device drivers that are closed source, or are simply not part of the actively-maintained kernel source tree.
vxNsr 6y ago
I always thought that was kinda the point of an OS, it differentiates itself based on how it does drivers and hardware management, it wraps it all in neat apis so that developers can focus on making over-engineered resource intensive chat clients.
- Koshkin 6y ago
  > the point of an OS, it differentiates itself based on how it does drivers Not really; also, the freedom to do something "different" in the device driver space is extremely limited: all you can do is decide on the packaging, kernel space vs. userspace and not much more.
  - spijdar 6y ago
    I think what GP was getting at is a big part of an OS is resource management (process permissions and isolation) and providing abstract interfaces over hardware (filesystems vs raw block storage). So beyond the low level drivers that directly interact with hardware, an OS differs in how you ... drive the drivers. What your HAL looks like, and then what the abstractions above that look like. It's by that definition that e.g. Android is considered a different OS from GNU/Linux. Same kernel, but the hardware is exposed much differently to end-user applications.
ronsor 6y ago
UEFI sort of handles this, even if the spec is absurdly huge. It contains portions referring to standardized ways of interacting with Wi-Fi cards and other devices, even if not all firmware supports it. I started an OS project[1] a while ago based on it. [1] https://github.com/HidamariProject/Hidamari https://github.com/HidamariProject/Hidamari
jonjacky 6y ago
> I wish there was a nice wrapper dedicated to abstracting hardware support There is NopSys, about 3500 lines of C and 600 lines of assembler for booting, accessing device registers, and responding to interrupts. So you can write the rest of the OS on top of that. CogNOS is a Smalltalk system written on top of NopSys. https://github.com/nopsys/nopsys https://github.com/nopsys/nopsys https://github.com/nopsys/CogNOS https://github.com/nopsys/CogNOS https://charig.github.io/assets/papers/SCDE-DLS.pdf https://charig.github.io/assets/papers/SCDE-DLS.pdf
- scroot 6y ago
  Do you have an update on the status of this work? Seems like it hasn't been worked on recently
cfallin 6y ago
> I wish there was a nice wrapper dedicated to abstracting hardware support The NetBSD kernel has the ability to be built as a library, letting various drivers be used in other ways -- it's called a "rump kernel" [0]. I've never played with it myself but my understanding is that this would be sufficient to provide drivers in one's own kernel, given enough plumbing and shims and such. (With enough shims one could borrow drivers from any kernel -- see ndiswrapper to use Windows network-card drivers in Linux -- but the rump kernel scaffolding is supposed to make it easier.) [0] https://www.netbsd.org/gallery/presentations/justin/2015_AsiaBSDCon/justincormack-abc2015.pdf https://www.netbsd.org/gallery/presentations/justin/2015_Asi...
m463 6y ago
I've wondered about this and thought - wouldn't this be easier with two very early decisions about the OS: - OO support / modularity - microkernel If this decision was made and adhered to early then: - new drivers could just import and inherit, filling in only the specifics - new drivers would be self-contained and (hopefully) not affect stability of the system I guess the problem is that most drivers have already been implemented in C and would require redesign or at least porting.

cjdell 6y ago

Completely agree. Something like Project Treble, but for PC hardware. Thinking about it, this is what the BIOS was originally for: to provide a bare minimum abstraction to the hardware. I hoped that UEFI would extend this abstraction in a meaningful way, making OS development easier. Instead it has made it even more complex.

hapless 6y ago
The original EFI spec very much did extend that abstraction. You can do a lot in an EFI environment if the hardware plays ball: network, disk, video etc UEFI, sadly, added support for raw x86 option ROMs, to some extent casting us back into the "old" world.

userbinator 6y ago

In the earlier days of the x86 PC, the hardware was far more documented, but now a lot of manufacturers are notoriously secretive about it. Nonetheless, standards still exist (HDA,ATA/ATAPI/SATA,OHCI/UHCI/EHCI/XHCI,etc.) and much of the backwards compatibility is still there. You bring up wi-fi cards and that is definitely one category of hardware with little standardisation/documentation (I suspect due to FCC regulations and such.) Wired ethernet is a bit better --- Realtek and Intel NICs are both quite well-documented, with the latter being more complex. The other difficult one is GPU acceleration (but a VESA framebuffer is usually available, which gets you at least a GUI if not a fast one.)

Koshkin 6y ago

> a nice wrapper Are Linux device drivers not nice enough? Hell, why couldn’t one think, for that matter, of a minimal Linux kernel as some kind of BIOS. Incidentally, we already use it as a hypervisor, so you could probably even include support for the devices you want in the hypervisor itself.

juancampa 6y ago
That's the approach urbit has taken
tenebrisalietum 6y ago
There are projects that load Linux as a UEFI DXE, and coreboot pretty much can be configured to initialize the platform and then hand over to a Linux payload immediately.

skissane 6y ago

> I wish there was a nice wrapper dedicated to abstracting hardware support better so developers can focus on the interesting parts The University of Utah had a research project back in the late 1990s which offered exactly that – the OSKit project [0]. A set of libraries for common (at the time) x86 hardware so OS developers and researchers could focus on what was original about their OS instead of writing yet another hard disk or Ethernet driver Unfortunately, the research project ended circa 2000 and it is dead now. No support for contemporary devices like Wifi and USB. But, if you just want something to run in a VM (QEMU or VirtualBox or whatever) it probably still works. [0] https://www.cs.utah.edu/flux/oskit/ https://www.cs.utah.edu/flux/oskit/

tooltower 6y ago
Today, when I build a custom Linux kernel, and am playing with config vars to decide what is our isn't included in the build, I very much tend to think of it as assembling a kernel from various library pieces. How was OSkit different?
- skissane 6y ago
  Linux isn't written with the use case of using bits of it in isolation in mind. Sure, you can do that, but the Linux developers don't have making that easy as a design goal, so it will often be quite hard, and even when it isn't so hard that is just an accident. By contrast, the OSKit developers had as an explicit design goal to make it easy to use bits of their code in isolation from the rest. Linux gives no API stability guarantees for its internal APIs, only for the system call interface. By contrast, the OSKit developers intended the API of each component to be a stable documented interface. The Linux developers don't want Linux's internal APIs to be stable documented interfaces because that has a cost in that it slows down refactoring, which in turn reduces the various benefits that refactoring can produce (better performance, better maintainability, new features, etc)
  - octorian 6y ago
    > The Linux developers don't want Linux's internal APIs to be stable documented interfaces It also has the "benefit" of making it extremely difficult to actually distribute stable device drivers that are closed source, or are simply not part of the actively-maintained kernel source tree.

vxNsr 6y ago

I always thought that was kinda the point of an OS, it differentiates itself based on how it does drivers and hardware management, it wraps it all in neat apis so that developers can focus on making over-engineered resource intensive chat clients.

Koshkin 6y ago
> the point of an OS, it differentiates itself based on how it does drivers Not really; also, the freedom to do something "different" in the device driver space is extremely limited: all you can do is decide on the packaging, kernel space vs. userspace and not much more.
- spijdar 6y ago
  I think what GP was getting at is a big part of an OS is resource management (process permissions and isolation) and providing abstract interfaces over hardware (filesystems vs raw block storage). So beyond the low level drivers that directly interact with hardware, an OS differs in how you ... drive the drivers. What your HAL looks like, and then what the abstractions above that look like. It's by that definition that e.g. Android is considered a different OS from GNU/Linux. Same kernel, but the hardware is exposed much differently to end-user applications.

ronsor 6y ago

UEFI sort of handles this, even if the spec is absurdly huge. It contains portions referring to standardized ways of interacting with Wi-Fi cards and other devices, even if not all firmware supports it. I started an OS project[1] a while ago based on it. [1] https://github.com/HidamariProject/Hidamari https://github.com/HidamariProject/Hidamari

jonjacky 6y ago

> I wish there was a nice wrapper dedicated to abstracting hardware support There is NopSys, about 3500 lines of C and 600 lines of assembler for booting, accessing device registers, and responding to interrupts. So you can write the rest of the OS on top of that. CogNOS is a Smalltalk system written on top of NopSys. https://github.com/nopsys/nopsys https://github.com/nopsys/nopsys https://github.com/nopsys/CogNOS https://github.com/nopsys/CogNOS https://charig.github.io/assets/papers/SCDE-DLS.pdf https://charig.github.io/assets/papers/SCDE-DLS.pdf

scroot 6y ago
Do you have an update on the status of this work? Seems like it hasn't been worked on recently

cfallin 6y ago

> I wish there was a nice wrapper dedicated to abstracting hardware support The NetBSD kernel has the ability to be built as a library, letting various drivers be used in other ways -- it's called a "rump kernel" [0]. I've never played with it myself but my understanding is that this would be sufficient to provide drivers in one's own kernel, given enough plumbing and shims and such. (With enough shims one could borrow drivers from any kernel -- see ndiswrapper to use Windows network-card drivers in Linux -- but the rump kernel scaffolding is supposed to make it easier.) [0] https://www.netbsd.org/gallery/presentations/justin/2015_AsiaBSDCon/justincormack-abc2015.pdf https://www.netbsd.org/gallery/presentations/justin/2015_Asi...

m463 6y ago

I've wondered about this and thought - wouldn't this be easier with two very early decisions about the OS: - OO support / modularity - microkernel If this decision was made and adhered to early then: - new drivers could just import and inherit, filling in only the specifics - new drivers would be self-contained and (hopefully) not affect stability of the system I guess the problem is that most drivers have already been implemented in C and would require redesign or at least porting.