4 ms·

This is one of the first papers introducing exokernels. The paper argues for breaking the shackles of OS abstraction and giving direct, (but mediated), control

by jws 11y ago

This is one of the first papers introducing exokernels. The paper argues for breaking the shackles of OS abstraction and giving direct, (but mediated), control to the programs. I like the idea, and am experimenting with it in my own OS. In the mainstream world, we see this paper's goals partially appearing with the people who give a network card directly to a server process in Linux. If they said which architecture there were thinking about, I missed it. On x86 without IOMMU I can't think of any card I would trust. IOMMU implementation is so spotty I'm not sure I'd trust it either. There is a great deal to be said for keeping the device access locked away in the kernel where you know which features of the card you will use, and if it has a mode that lets it scribble elsewhere in memory, so aren't using it.

ykumar6 11y ago
Yes, but do software companies really care about "giving direct, (but mediated), control to the programs". What is the trade-off between that, and the added complexity in understanding all the different ways you can program something? IMO, developers want a simple and standardized model to build applications on operating systems so they don't have to care about the internals. This article represents a marginalized view by the larger industry as a whole.
- dang 11y ago
  When the authors talk about applications having their own OS, I think they have in mind infrastructure programs like databases and web servers: complex, low-level programs that have domain-specific needs from the hardware and can gain greatly by using it only in the way that they need to. They obviously don't mean that every CRUD app should build its own OS, since that would be absurd.
  - eru 11y ago
    But every CRUD app can links in its own OS as a standardized library.
  - greenyoda 11y ago
    A database is a good example. For example, a database engine might benefit from being able to read large raw blocks from a disk and directly optimizing its own disk seeks, rather than using the common abstraction of a file as being just a stream of bytes.
    - nickpsecurity 11y ago
      You're right: it was proven with the invention of object-based, disk storage that kept things simple. Think NoSQL key-value stores vs common SQL databases in performance. Eliminating complexity in the underlying mechanism while allowing the app to decide how it's used can certainly benefit performance.
- panic 11y ago
  Nobody is saying we should write all our programs directly against hardware. The point of the paper is that the operating system should let applications define their own abstractions. This is already standard practice for the CPU: the OS lets applications run machine code directly, and you can use whatever language you'd like to generate that machine code. You can still have a standardized model for writing software. The argument is against baking this standardized model into the kernel, where it will stick around for decades, unable to be changed.
  - ykumar6 11y ago
    Yes, I 100% understand that point. But I repeat what I'm saying, this isn't a concern for most developers and most software companies. They don't care how a page table is written. They don't care about optimizing that. Every standard or abstraction can be unbundled into multiple standards or abstractions. But at the end of the day, what wins is something that's simplest to the operator
    - vezzy-fnord 11y ago
      You're not really contradicting anything, you're just making an argument from personal incredulity. People do care, you may not. Else the Linux kernel would have had a fraction of its manpower and no one would want to introduce userspace page fault handling (just recently userfaultfd(2) went in from a KVM optimization effort) or use userland TCP/IP.
- anon4 11y ago
  I agree with you, and here is one more concern: a lot of hardware can't be shared at the hardware level. Think of the soundcard for example. A lot of PCs have simple Intel-HDA audio controllers that don't have hardware-level separate streams. You need a software defined mixer that will share the resource with several applications. For years and years on Linux the situation was that you do get direct hardware access and it sucked balls. Either your card has hardware mixing and you can play like 8 sounds at once, or it doesn't and you can't. In that time, at least 3 different userspace mixers appeared, all incompatible with each other, which of course means that your choice of Gnome vs KDE also dictates which music players you can use. The OS absolutely needs to define standard usage APIs so we don't end up in the situation where everybody rolls their own. Yes, giving applications bare-metal access is good. But that's a minority of applications in a minority of usage cases. Give people a good simple API for the average case, and THEN let them take exclusive low-level control when they need it.
fulafel 11y ago
IOMMUs might have been standard fare in the workstation hardware of the day. SPARCs had them at least.
- nickpsecurity 11y ago
  SCOMP, A1 secure system of 19??-1985, invented them far as I know. It's one of those old concepts that was ignored repeatedly and re-discovered. Thing is, they left off most of the other good feature and assurance activities Orange Book taught. They're in the process of slowly re-discovering those with "timing-related, side channels" being one. Early Orange Book candidates knocked out "covert timing channels" with mandated, covert, channel analysis back in the 1980's-90's, including for virtualization. I'm just waiting for the rest to be re-discovered by mainstream so we can get on with it and build a secure platform with significant developer support. SCOMP Final Report (see p17 of PDF or p9 of document on I/O) http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA229523 http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDo...

by jws 11y ago

ykumar6 11y ago

Yes, but do software companies really care about "giving direct, (but mediated), control to the programs". What is the trade-off between that, and the added complexity in understanding all the different ways you can program something? IMO, developers want a simple and standardized model to build applications on operating systems so they don't have to care about the internals. This article represents a marginalized view by the larger industry as a whole.

dang 11y ago
When the authors talk about applications having their own OS, I think they have in mind infrastructure programs like databases and web servers: complex, low-level programs that have domain-specific needs from the hardware and can gain greatly by using it only in the way that they need to. They obviously don't mean that every CRUD app should build its own OS, since that would be absurd.
- eru 11y ago
  But every CRUD app can links in its own OS as a standardized library.
- greenyoda 11y ago
  A database is a good example. For example, a database engine might benefit from being able to read large raw blocks from a disk and directly optimizing its own disk seeks, rather than using the common abstraction of a file as being just a stream of bytes.
  - nickpsecurity 11y ago
    You're right: it was proven with the invention of object-based, disk storage that kept things simple. Think NoSQL key-value stores vs common SQL databases in performance. Eliminating complexity in the underlying mechanism while allowing the app to decide how it's used can certainly benefit performance.
panic 11y ago
Nobody is saying we should write all our programs directly against hardware. The point of the paper is that the operating system should let applications define their own abstractions. This is already standard practice for the CPU: the OS lets applications run machine code directly, and you can use whatever language you'd like to generate that machine code. You can still have a standardized model for writing software. The argument is against baking this standardized model into the kernel, where it will stick around for decades, unable to be changed.
- ykumar6 11y ago
  Yes, I 100% understand that point. But I repeat what I'm saying, this isn't a concern for most developers and most software companies. They don't care how a page table is written. They don't care about optimizing that. Every standard or abstraction can be unbundled into multiple standards or abstractions. But at the end of the day, what wins is something that's simplest to the operator
  - vezzy-fnord 11y ago
    You're not really contradicting anything, you're just making an argument from personal incredulity. People do care, you may not. Else the Linux kernel would have had a fraction of its manpower and no one would want to introduce userspace page fault handling (just recently userfaultfd(2) went in from a KVM optimization effort) or use userland TCP/IP.
anon4 11y ago
I agree with you, and here is one more concern: a lot of hardware can't be shared at the hardware level. Think of the soundcard for example. A lot of PCs have simple Intel-HDA audio controllers that don't have hardware-level separate streams. You need a software defined mixer that will share the resource with several applications. For years and years on Linux the situation was that you do get direct hardware access and it sucked balls. Either your card has hardware mixing and you can play like 8 sounds at once, or it doesn't and you can't. In that time, at least 3 different userspace mixers appeared, all incompatible with each other, which of course means that your choice of Gnome vs KDE also dictates which music players you can use. The OS absolutely needs to define standard usage APIs so we don't end up in the situation where everybody rolls their own. Yes, giving applications bare-metal access is good. But that's a minority of applications in a minority of usage cases. Give people a good simple API for the average case, and THEN let them take exclusive low-level control when they need it.

fulafel 11y ago

IOMMUs might have been standard fare in the workstation hardware of the day. SPARCs had them at least.

nickpsecurity 11y ago
SCOMP, A1 secure system of 19??-1985, invented them far as I know. It's one of those old concepts that was ignored repeatedly and re-discovered. Thing is, they left off most of the other good feature and assurance activities Orange Book taught. They're in the process of slowly re-discovering those with "timing-related, side channels" being one. Early Orange Book candidates knocked out "covert timing channels" with mandated, covert, channel analysis back in the 1980's-90's, including for virtualization. I'm just waiting for the rest to be re-discovered by mainstream so we can get on with it and build a secure platform with significant developer support. SCOMP Final Report (see p17 of PDF or p9 of document on I/O) http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA229523 http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDo...