6 ms·

Barrelfish OS Architecture Overview (2013) [pdf]

peter_d_sherman 1y ago
This is a very interesting OS design: >"1.1 High level overview Barrelfish is “multikernel” operating system [3]: it consists of a small kernel running on each core (one kernel per core), and while rest of the OS is structured as a distributed system of single-core processes atop these kernels. Kernels share no memory, even on a machine with cache-coherent shared RAM, and the rest of the OS does not use shared memory except for transferring messages and data between cores, and booting other cores."
- transpute 1y ago
  2021 Usenix talk by Barrelfish researcher, "It's Time for Operating Systems to Rediscover Hardware", 90 comments, https://news.ycombinator.com/item?id=28374523 https://news.ycombinator.com/item?id=28374523
- kfreds 1y ago
  You might also find these interesting: - Akaros, an OS for manycore systems: http://akaros.org/news.html http://akaros.org/news.html - VMThreads, an interesting paper on scheduling challenges, related to Akaros: https://iwp9.org/11iwp9proceedings.pdf https://iwp9.org/11iwp9proceedings.pdf
- jdefr89 1y ago
  Was I the only one confused by this? It wasn't just me right? I love when I see things like this. "The cool thing about our kernel is that you cannot share memory! It's super secure. Except for, you know, ..." then list nearly everything. What were they trying to provide/gain with this proposal?
andsoitis 1y ago
and you can download and run it: https://barrelfish.org/download.html https://barrelfish.org/download.html
- binarycrusader 1y ago
  sadly, last release 2020-03-23: The Barrelfish project is no longer active. See https://systems.ethz.ch/ https://systems.ethz.ch/ for information about our current research activities. It is still interesting though.
  - transpute 1y ago
    They have moved onto RISC-V hardware for manycore architecture, https://news.ycombinator.com/item?id=43629804 https://news.ycombinator.com/item?id=43629804 Tenstorrent (Jim Keller) shipped RISC-V manycore design for inference.
    - panick21_ 1y ago
      No this group actually mostly works on: https://enzian.systems/ https://enzian.systems/
      - transpute 1y ago
        Thanks for the correction.
- transpute 1y ago
  Ten years of OS research, supporting x86, ARMv7 and ARMv8 devices, leading to 2021 talk about hardware and subsequent design of new hardware (RISC-V).
transpute 1y ago
Systems research geneaology: Xen [U of Cambridge, XenSource, Citrix] KVM [Qumranet, RedHat] EC2 [AWS] Nitro [Annapurna Labs, AWS] Barrelfish [ETH Zurich, Microsoft] Snitch RISC-V (many)core uXen ("micro" Xen, CoW memory) [Bromium, HP] firecracker AX x86 ("atto" Xen) [Bromium, HP] pKVM Arm [Google, Android, Linux]
- kfreds 1y ago
  Interesting. Do you know of any good SoK papers or articles that summarize the current state of the art, or explains this genealogy?
  - transpute 1y ago
    A longer history would start with IBM mainframes. More recently, IBM Ultravisor shipped in OpenPower firmware, mediating KVM VMs, https://www.youtube.com/watch?v=6qjrqn3ug0g https://www.youtube.com/watch?v=6qjrqn3ug0g & https://github.com/open-power/ultravisor https://github.com/open-power/ultravisor 2018 video by Ian Pratt covers Xen, uXen and AX (2005-2015), https://news.ycombinator.com/item?id=44135977#44141164 https://news.ycombinator.com/item?id=44135977#44141164. Citrix acquired XenSource. Pratt left to work at Bromium, acquired by HP (which previously acquired BIOS company from Bromium co-founder). The former CTO of XenSource co-founded Qumranet (KVM), acquired by RedHat. AWS began with Xen, then migrated to a subset of KVM. Nitro used Arm hardware to virtualize I/O (storage, network) paths, leaving KVM responsible for x86 CPU and memory virtualization, https://www.youtube.com/watch?v=e8DVmwj3OEs https://www.youtube.com/watch?v=e8DVmwj3OEs & https://news.ycombinator.com/item?id=24515019#24516523 https://news.ycombinator.com/item?id=24515019#24516523. Parallels could be drawn to the Apple T2 enclave (Arm) coprocessor being used for disk encryption on x86 Apple Macbooks. Under the "Confidential Computing" umbrella, Intel has TDX and a new (closed?) hypervisor on servers, using SGX and new hardware privilege levels. Apple recently added Secure eXclaves to iOS, and Apple Silicon hardware supports nested virtualization, which is what Google pKVM uses on Pixel (and upcoming ChromeOS?) devices, https://news.ycombinator.com/item?id=43314657 https://news.ycombinator.com/item?id=43314657 For production code, pKVM deserves attention because it's open (upstreamed to mainline Linux), exists in the real world (Pixel phones), stands in stark contrast to Apple's neutered iPads and has the potential to improve upon TrustZone security, https://news.ycombinator.com/item?id=41523758 https://news.ycombinator.com/item?id=41523758. Finally, to bring this thread back to Barrelfish, Google OpenTitan open silicon root of trust (OCP servers, Chromebooks) is partly under Pulp Platform research, alongside Snitch (descended from Barrelfish research) open hardware from ETH Zurich. So progress is being made in both mainstream-compatible systems software and greenfield hardware cores. (hopefully readers can correct any errors or gaps above)
davemp 1y ago
I find these type of efforts somewhat disappointing. So much OS research boils down to “We’ll handle scheduling and rudimentary peripheral multiplexing good luck on rest”. These basics are so far from a useful system that you’d have to slap linux on top and immediately lose most/all benefits of the new architecture.
- yencabulator 1y ago
  It's a research project. It has been influential enough. Researchers have also made hardware prototypes that pushed this message-passing-cores design ever further. In the non-academic world, HPC and realtime people have reimplemented some of these ideas into Linux, making it so that a core can be fully dedicated to an application, not receiving interrupts ("tickless"), not handling any kernel tasks, etc. For example, https://htor.inf.ethz.ch/ross2012/slides/ross2012-akkan.pdf https://htor.inf.ethz.ch/ross2012/slides/ross2012-akkan.pdf https://insidehpc.com/2009/10/tilera-100-core-x86-architecture/ https://insidehpc.com/2009/10/tilera-100-core-x86-architectu... https://lwn.net/Articles/549580/ https://lwn.net/Articles/549580/ https://lwn.net/Articles/816298/ https://lwn.net/Articles/816298/

This is a very interesting OS design: >"1.1 High level overview Barrelfish is “multikernel” operating system [3]: it consists of a small kernel running on each core (one kernel per core), and while rest of the OS is structured as a distributed system of single-core processes atop these kernels. Kernels share no memory, even on a machine with cache-coherent shared RAM, and the rest of the OS does not use shared memory except for transferring messages and data between cores, and booting other cores."

transpute 1y ago
2021 Usenix talk by Barrelfish researcher, "It's Time for Operating Systems to Rediscover Hardware", 90 comments, https://news.ycombinator.com/item?id=28374523 https://news.ycombinator.com/item?id=28374523
kfreds 1y ago
You might also find these interesting: - Akaros, an OS for manycore systems: http://akaros.org/news.html http://akaros.org/news.html - VMThreads, an interesting paper on scheduling challenges, related to Akaros: https://iwp9.org/11iwp9proceedings.pdf https://iwp9.org/11iwp9proceedings.pdf
jdefr89 1y ago
Was I the only one confused by this? It wasn't just me right? I love when I see things like this. "The cool thing about our kernel is that you cannot share memory! It's super secure. Except for, you know, ..." then list nearly everything. What were they trying to provide/gain with this proposal?

andsoitis 1y ago

and you can download and run it: https://barrelfish.org/download.html https://barrelfish.org/download.html

binarycrusader 1y ago
sadly, last release 2020-03-23: The Barrelfish project is no longer active. See https://systems.ethz.ch/ https://systems.ethz.ch/ for information about our current research activities. It is still interesting though.
- transpute 1y ago
  They have moved onto RISC-V hardware for manycore architecture, https://news.ycombinator.com/item?id=43629804 https://news.ycombinator.com/item?id=43629804 Tenstorrent (Jim Keller) shipped RISC-V manycore design for inference.
  - panick21_ 1y ago
    No this group actually mostly works on: https://enzian.systems/ https://enzian.systems/
    - transpute 1y ago
      Thanks for the correction.
transpute 1y ago
Ten years of OS research, supporting x86, ARMv7 and ARMv8 devices, leading to 2021 talk about hardware and subsequent design of new hardware (RISC-V).

transpute 1y ago

Systems research geneaology: Xen [U of Cambridge, XenSource, Citrix] KVM [Qumranet, RedHat] EC2 [AWS] Nitro [Annapurna Labs, AWS] Barrelfish [ETH Zurich, Microsoft] Snitch RISC-V (many)core uXen ("micro" Xen, CoW memory) [Bromium, HP] firecracker AX x86 ("atto" Xen) [Bromium, HP] pKVM Arm [Google, Android, Linux]

kfreds 1y ago
Interesting. Do you know of any good SoK papers or articles that summarize the current state of the art, or explains this genealogy?
- transpute 1y ago
  A longer history would start with IBM mainframes. More recently, IBM Ultravisor shipped in OpenPower firmware, mediating KVM VMs, https://www.youtube.com/watch?v=6qjrqn3ug0g https://www.youtube.com/watch?v=6qjrqn3ug0g & https://github.com/open-power/ultravisor https://github.com/open-power/ultravisor 2018 video by Ian Pratt covers Xen, uXen and AX (2005-2015), https://news.ycombinator.com/item?id=44135977#44141164 https://news.ycombinator.com/item?id=44135977#44141164. Citrix acquired XenSource. Pratt left to work at Bromium, acquired by HP (which previously acquired BIOS company from Bromium co-founder). The former CTO of XenSource co-founded Qumranet (KVM), acquired by RedHat. AWS began with Xen, then migrated to a subset of KVM. Nitro used Arm hardware to virtualize I/O (storage, network) paths, leaving KVM responsible for x86 CPU and memory virtualization, https://www.youtube.com/watch?v=e8DVmwj3OEs https://www.youtube.com/watch?v=e8DVmwj3OEs & https://news.ycombinator.com/item?id=24515019#24516523 https://news.ycombinator.com/item?id=24515019#24516523. Parallels could be drawn to the Apple T2 enclave (Arm) coprocessor being used for disk encryption on x86 Apple Macbooks. Under the "Confidential Computing" umbrella, Intel has TDX and a new (closed?) hypervisor on servers, using SGX and new hardware privilege levels. Apple recently added Secure eXclaves to iOS, and Apple Silicon hardware supports nested virtualization, which is what Google pKVM uses on Pixel (and upcoming ChromeOS?) devices, https://news.ycombinator.com/item?id=43314657 https://news.ycombinator.com/item?id=43314657 For production code, pKVM deserves attention because it's open (upstreamed to mainline Linux), exists in the real world (Pixel phones), stands in stark contrast to Apple's neutered iPads and has the potential to improve upon TrustZone security, https://news.ycombinator.com/item?id=41523758 https://news.ycombinator.com/item?id=41523758. Finally, to bring this thread back to Barrelfish, Google OpenTitan open silicon root of trust (OCP servers, Chromebooks) is partly under Pulp Platform research, alongside Snitch (descended from Barrelfish research) open hardware from ETH Zurich. So progress is being made in both mainstream-compatible systems software and greenfield hardware cores. (hopefully readers can correct any errors or gaps above)

davemp 1y ago

I find these type of efforts somewhat disappointing. So much OS research boils down to “We’ll handle scheduling and rudimentary peripheral multiplexing good luck on rest”. These basics are so far from a useful system that you’d have to slap linux on top and immediately lose most/all benefits of the new architecture.

yencabulator 1y ago
It's a research project. It has been influential enough. Researchers have also made hardware prototypes that pushed this message-passing-cores design ever further. In the non-academic world, HPC and realtime people have reimplemented some of these ideas into Linux, making it so that a core can be fully dedicated to an application, not receiving interrupts ("tickless"), not handling any kernel tasks, etc. For example, https://htor.inf.ethz.ch/ross2012/slides/ross2012-akkan.pdf https://htor.inf.ethz.ch/ross2012/slides/ross2012-akkan.pdf https://insidehpc.com/2009/10/tilera-100-core-x86-architecture/ https://insidehpc.com/2009/10/tilera-100-core-x86-architectu... https://lwn.net/Articles/549580/ https://lwn.net/Articles/549580/ https://lwn.net/Articles/816298/ https://lwn.net/Articles/816298/