8 ms·

IBM CTO Sees Smaller Container Platforms Ahead

70 points by eyberg 7y ago

atq2119 7y ago
Title doesn't match the article and contradicts the article's contents. The article talks about using unikernels with containers, not instead of them. BTW, I kind of assume they mean running unikernels inside containers as opposed to running containers on unikernels. The article seems a bit self-contradictory about that.
- eyberg 7y ago
  IBM/redhat have a few unikernel related projects going on and I'm guessing this is referring to nabla containers (or he might be aware of ukl). The only reason to stuff an unikernel into a container is to make the transition easier for k8s/container holdouts. You'll take a perf hit and still be stuck with all the security problems containers have. Other than that at least in my incredibly biased opinion it makes no sense.
- Matthias247 7y ago
  Quote from the article: > At the same time, many containers in the years ahead will be deployed on unikernel systems, says Ferris I however can't see how this would make sense.
rwmj 7y ago
[Disclaimer: I work for Red Hat in this area, but don't speak for IBM or their CTO] I think reading between the lines this is really about Kata Containers-style fast booting virtual machines which run containers and give you a bit more security. Development of this is very far along, expect to see this being deployed widely soon by enterprises and others who have security sensitive container workloads. The unikernels stuff seems rather more speculative, and I don't know specifically what IBM is referring to, but I am [at Red Hat] working with a student at BU on turning Linux into a unikernel. You link your application directly to Linux and get a vmlinuz binary which can be run in a variety of places including on cloud VMs, in qemu, in a Kata Container [in theory - not actually tried this one], or even on baremetal. Old article here: https://next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-linux/ https://next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-... Edit: I should say for completeness there is also KubeVirt, which lets you run your pet VMs in a pod on Kubernetes. Sort of the opposite of Kata.
- angrygoat 7y ago
  Is what you're doing anything like MirageOS (NetBSD rump kernel linked with an OCaml application [Note, this was an error, see below])? It sounds really interesting – I wonder what advantages Linux vs *BSD have as unikernels, when most hardware details are out of the picture.
  - justincormack 7y ago
    MirageOS does not link to the NetBSD rump kernel. It is a standalone unikernel.
    - angrygoat 7y ago
      Thanks for the correction :)
- pnako 7y ago
  So if I understand correctly, you end up with some sort of binary, that you can run on top of some system that abstracts away the hardware and manages resources, correct?
  - rwmj 7y ago
    You end up with a self-contained application binary which runs anywhere that you could run a Linux kernel. However it's different from embedding an initrd into a kernel image in several ways: The kernel is minimal, containing only the bits which are needed by your application or drivers required to run on the hardware. The application runs in kernel space and is linked with the kernel which gives you a nice performance boost because you never context switch (which is the driver behind the whole project actually). You can also call kernel functions which aren't syscalls, eg. for specialized memory management.
    - 0x445442 7y ago
      64-bit DOS?
kkwak 7y ago
containers are implemented via cgroups (and others) and share the kernel. I presume with a unikernel, you'll not be sharing the kernel, obviously.. So concept of a "container" is there and probably the orchestration via kubernetes is all still valid up the stack, so it makes sense to call that "containers" still.
- AzzieElbab 7y ago
  Unikernel should run directly on hypervisor. Not sure this plays along with multiple levels of virtualization in today's clouds.
  - eyberg 7y ago
    It does to a degree but there is immense confusion on how it does. For example with OPS https://ops.city https://ops.city (I'm associated with the project) we deploy directly to AWS or GCloud as an ami or cloud image not orchestrating qemu on top of an existing linux image. A lot of people until they have actually tried booting one are thoroughly confused on this process. You could play on top of google nested virt or ami's 'bare metal' but both of those would have performance taxes. I think in the coming years we'll see the big public clouds refactor their environments to support these in a better fashion.
moomin 7y ago
Erm... this is corporate thinkpiece nonsense.
pjmlp 7y ago
Same here, serveless computing and rich language runtimes don't care if they are running on top of an OS, container, or even bare metal. "The Cloud" will drive the OS layer away into plain cloud services, and the mainframe language environments will be back.
- tybit 7y ago
  I agree. I wonder whether they end up being language agnostic or not.
  - 0x445442 7y ago
    They sort of already are: https://github.com/newapplesho/aws-sdk-smalltalk https://github.com/newapplesho/aws-sdk-smalltalk
mmillin 7y ago
> These advancements will be occurring in parallel to the rise of immutable databases in the form of blockchain platforms, applications capable of invoking quantum computing systems and artificial intelligence (AI) platforms, which are not only going to become more accessible but also certified to be trusted and secure, says Ferris. Article just seems buzzword soup, bringing up: Containers, Unikernels, Blockchain, AI, 5G, IoT, Edge Computing, Quantum Computing > containers in the years ahead will be deployed on unikernel systems What does this even mean? Containers are built around sharing a kernel, and unikernels are all about having a kernel custom for your application. Seems the ideas are at odds. The most charitable view to me seems to be an orchestration platform for running "containers" which are actually VMs running unikernels, instead of contained processes?
- justincormack 7y ago
  I don't think containers are defined by having a shared kernel. VM based containers, with the same API, have been around for a few years now. Cloud services serve up large numbers of these (see Fargate etc; many of these run a container inside a VM but its much the same).
- nordsieck 7y ago
  > What does this even mean? Containers are built around sharing a kernel, and unikernels are all about having a kernel custom for your application. Seems the ideas are at odds. I think the most charitable translation of this is that the OS inside the container (e.g. Alpine Linux) + application will be replaced with a single unified OS + application i.e. unikernel. I don't think that this is true, but it's a seductive idea. From what I can tell, as system complexity goes up, you need increased instrumentation in order to property debug your application. That is a lot of what modern OS's provide. I do sympathize with people making efficiency arguments, but I always try to bet on the "strong horse". There is just so much money and effort going into Linux, that I think it'll get efficient faster than unikernels will get instrumented.
  - baq 7y ago
    "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." putting an app in a unikernel may be about as clever as possible for a great many software engineers.
Edmond 7y ago
3...2...1...and..a new hype cycle begins...or at least IBM hopes it does.
sys_64738 7y ago
Is this below saying the kernel context executes within the container? A Hypervisor being communicated to via each container is a security nightmare. The idea is to isolate the container so it can't make any type of hypervisor calls by not making the HV API available to the running container. "Unikernels consist of a specialized, single-address-space machine image constructed using only a modular stack of libraries required to run an application. That approach allows a secure, fixed-purpose image to run directly on a hypervisor or bare-metal hardware without an operating system such as Linux or Windows. "
deleted 7y ago
[deleted]
AzzieElbab 7y ago
Personally I would never bet on anything alongside IBM, but this is interesting
duelingjello 7y ago
Nope, nope and nope. This fad is as tone-deaf as Howard Schultz running for POTUS. Unikernels are TERRIBLE for operations. How do you have privilege separation? Separation of concerns? Processes? Users? Groups? Talk to a database or LDAP? Have a reliable filesystem? Drivers for hardware? Updates? Security audits? They’re the “emperor’s new clothes” but they throw away ALL the generality, security, lessons and existing infrastructure by reinventing the wheel, badly. They were a fad years ago but failed for these and many reasons.
- eyberg 7y ago
  Most unikernels dont have the notion of users, groups, or multiple processes on purpose for security and performance reasons. Some unikernels use the best in class file systems available like zfs by default. Containers have all of these. Containers aren't unikernels. Drivers are way less of a concern cause all you need is a clock, a network driver and a disk driver, not 30 different USB drivers, half a dozen networking drivers and more. Half of linux is just drivers cause it's designed to run on real machines whereas unikernels are explicitly designed to run as vms. Happy to engage with real life examples on these concerns but the best way to truly understand since there's all sorts of misunderstanding here is to just simply deploy one.
- sp332 7y ago
  For every single ? in your comment, the answer is "it's done at the hypervisor level". A unikernel is basically a single process, so privilege separation, users, groups etc are irrelevant. Drivers are much, much simpler (and I would argue, easier to audit) because they mostly only need to access virtualized, hypervisor-provided HAL APIs. Filesystem, DB, LDAP would be external - possibly provided by another VM on the same physical hardware.
mathattack 7y ago
Will they find a way to slip in the Watson brand? IBM is no longer in the innovation business. They (along with Red Hat) are in the audit business.
deleted 7y ago
[deleted]
zackmorris 7y ago
Quick question/idea about security: with all of the exploits today like Meltdown and Spectre, is anyone working on a truly secure container? Something like a public website running on an Intel processor, and/or maybe subdomains running on AMD and other mainstream CPUs. They would each be running some number of containers (up to say 1000 to start) and anyone could log in anonymously and run whatever they want in a minimal OS like Ubuntu. They would probably just have access to a small filesystem and maybe some open ports for HTTP or SSL or whatever. The challenge would be to break out and take over the host computer. If anyone does that, then the pwned container is flagged as not secure. The idea here is that until this happens, we can't really run a container and allow public access. I'd like containers to be as ubiquitous as websites running Javascript in the browser's sandbox so that we can host our own REPLs. If companies are serious about this, they should also consider providing bounties, say $100,000 guaranteeing that their container is secure for users. If you get hacked, you get the money. Without something like that, I'm afraid that I can't believe that containers are secure on faith alone.
- lossolo 7y ago
  This doesn't work like that. Containers (like docker, LXC and other based on cgroups etc) are isolated by the kernel but they share the same kernel. This means that kernel exploit will make all those containers insecure by your definition. If you go further and try to embed containers in hardware virtualization layer (so basically mini VMs) you still don't have any guarantees because from time to time new exploit emerges that allows to break out from guest OS to hypervisor. Then you have hardware layer and exploits based on speculative execution (Spectre, Meltdown and friends). Truly secure container doesn't exist. You would need some kind of mathematical proof for whole hardware stack and all communications between different hardware subsystems + whole kernel + container runtime.
  - zackmorris 7y ago
    Hmmm that doesn't inspire a lot of confidence. I'd maybe be fine with a VM then, if containers fundamentally can't be made secure. Short of a mathematical "proof" of security, the best I could come up with was to leave containers open for the hackers of the world to exploit. If a container survives, that's a fairly strong guarantee that it can't be escaped (for now). What I am getting at is: if we know that hackers can break out of containers, then how are we ok with shared hosting running in containers? Once one gets owned, it's trivial to break into the others. Edit: this might be a way to force maintainers to fix kernel exploits as well.

70 points by eyberg 7y ago

Title doesn't match the article and contradicts the article's contents. The article talks about using unikernels with containers, not instead of them. BTW, I kind of assume they mean running unikernels inside containers as opposed to running containers on unikernels. The article seems a bit self-contradictory about that.

eyberg 7y ago
IBM/redhat have a few unikernel related projects going on and I'm guessing this is referring to nabla containers (or he might be aware of ukl). The only reason to stuff an unikernel into a container is to make the transition easier for k8s/container holdouts. You'll take a perf hit and still be stuck with all the security problems containers have. Other than that at least in my incredibly biased opinion it makes no sense.
Matthias247 7y ago
Quote from the article: > At the same time, many containers in the years ahead will be deployed on unikernel systems, says Ferris I however can't see how this would make sense.

rwmj 7y ago

[Disclaimer: I work for Red Hat in this area, but don't speak for IBM or their CTO] I think reading between the lines this is really about Kata Containers-style fast booting virtual machines which run containers and give you a bit more security. Development of this is very far along, expect to see this being deployed widely soon by enterprises and others who have security sensitive container workloads. The unikernels stuff seems rather more speculative, and I don't know specifically what IBM is referring to, but I am [at Red Hat] working with a student at BU on turning Linux into a unikernel. You link your application directly to Linux and get a vmlinuz binary which can be run in a variety of places including on cloud VMs, in qemu, in a Kata Container [in theory - not actually tried this one], or even on baremetal. Old article here: https://next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-linux/ https://next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-... Edit: I should say for completeness there is also KubeVirt, which lets you run your pet VMs in a pod on Kubernetes. Sort of the opposite of Kata.

angrygoat 7y ago
Is what you're doing anything like MirageOS (NetBSD rump kernel linked with an OCaml application [Note, this was an error, see below])? It sounds really interesting – I wonder what advantages Linux vs *BSD have as unikernels, when most hardware details are out of the picture.
- justincormack 7y ago
  MirageOS does not link to the NetBSD rump kernel. It is a standalone unikernel.
  - angrygoat 7y ago
    Thanks for the correction :)
pnako 7y ago
So if I understand correctly, you end up with some sort of binary, that you can run on top of some system that abstracts away the hardware and manages resources, correct?
- rwmj 7y ago
  You end up with a self-contained application binary which runs anywhere that you could run a Linux kernel. However it's different from embedding an initrd into a kernel image in several ways: The kernel is minimal, containing only the bits which are needed by your application or drivers required to run on the hardware. The application runs in kernel space and is linked with the kernel which gives you a nice performance boost because you never context switch (which is the driver behind the whole project actually). You can also call kernel functions which aren't syscalls, eg. for specialized memory management.
  - 0x445442 7y ago
    64-bit DOS?

kkwak 7y ago

containers are implemented via cgroups (and others) and share the kernel. I presume with a unikernel, you'll not be sharing the kernel, obviously.. So concept of a "container" is there and probably the orchestration via kubernetes is all still valid up the stack, so it makes sense to call that "containers" still.

AzzieElbab 7y ago
Unikernel should run directly on hypervisor. Not sure this plays along with multiple levels of virtualization in today's clouds.
- eyberg 7y ago
  It does to a degree but there is immense confusion on how it does. For example with OPS https://ops.city https://ops.city (I'm associated with the project) we deploy directly to AWS or GCloud as an ami or cloud image not orchestrating qemu on top of an existing linux image. A lot of people until they have actually tried booting one are thoroughly confused on this process. You could play on top of google nested virt or ami's 'bare metal' but both of those would have performance taxes. I think in the coming years we'll see the big public clouds refactor their environments to support these in a better fashion.

moomin 7y ago

Erm... this is corporate thinkpiece nonsense.

pjmlp 7y ago

Same here, serveless computing and rich language runtimes don't care if they are running on top of an OS, container, or even bare metal. "The Cloud" will drive the OS layer away into plain cloud services, and the mainframe language environments will be back.

tybit 7y ago
I agree. I wonder whether they end up being language agnostic or not.
- 0x445442 7y ago
  They sort of already are: https://github.com/newapplesho/aws-sdk-smalltalk https://github.com/newapplesho/aws-sdk-smalltalk

mmillin 7y ago

> These advancements will be occurring in parallel to the rise of immutable databases in the form of blockchain platforms, applications capable of invoking quantum computing systems and artificial intelligence (AI) platforms, which are not only going to become more accessible but also certified to be trusted and secure, says Ferris. Article just seems buzzword soup, bringing up: Containers, Unikernels, Blockchain, AI, 5G, IoT, Edge Computing, Quantum Computing > containers in the years ahead will be deployed on unikernel systems What does this even mean? Containers are built around sharing a kernel, and unikernels are all about having a kernel custom for your application. Seems the ideas are at odds. The most charitable view to me seems to be an orchestration platform for running "containers" which are actually VMs running unikernels, instead of contained processes?

justincormack 7y ago
I don't think containers are defined by having a shared kernel. VM based containers, with the same API, have been around for a few years now. Cloud services serve up large numbers of these (see Fargate etc; many of these run a container inside a VM but its much the same).
nordsieck 7y ago
> What does this even mean? Containers are built around sharing a kernel, and unikernels are all about having a kernel custom for your application. Seems the ideas are at odds. I think the most charitable translation of this is that the OS inside the container (e.g. Alpine Linux) + application will be replaced with a single unified OS + application i.e. unikernel. I don't think that this is true, but it's a seductive idea. From what I can tell, as system complexity goes up, you need increased instrumentation in order to property debug your application. That is a lot of what modern OS's provide. I do sympathize with people making efficiency arguments, but I always try to bet on the "strong horse". There is just so much money and effort going into Linux, that I think it'll get efficient faster than unikernels will get instrumented.
- baq 7y ago
  "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." putting an app in a unikernel may be about as clever as possible for a great many software engineers.

Edmond 7y ago

3...2...1...and..a new hype cycle begins...or at least IBM hopes it does.

sys_64738 7y ago

Is this below saying the kernel context executes within the container? A Hypervisor being communicated to via each container is a security nightmare. The idea is to isolate the container so it can't make any type of hypervisor calls by not making the HV API available to the running container. "Unikernels consist of a specialized, single-address-space machine image constructed using only a modular stack of libraries required to run an application. That approach allows a secure, fixed-purpose image to run directly on a hypervisor or bare-metal hardware without an operating system such as Linux or Windows. "

deleted 7y ago

[deleted]

AzzieElbab 7y ago

Personally I would never bet on anything alongside IBM, but this is interesting

duelingjello 7y ago

Nope, nope and nope. This fad is as tone-deaf as Howard Schultz running for POTUS. Unikernels are TERRIBLE for operations. How do you have privilege separation? Separation of concerns? Processes? Users? Groups? Talk to a database or LDAP? Have a reliable filesystem? Drivers for hardware? Updates? Security audits? They’re the “emperor’s new clothes” but they throw away ALL the generality, security, lessons and existing infrastructure by reinventing the wheel, badly. They were a fad years ago but failed for these and many reasons.

eyberg 7y ago
Most unikernels dont have the notion of users, groups, or multiple processes on purpose for security and performance reasons. Some unikernels use the best in class file systems available like zfs by default. Containers have all of these. Containers aren't unikernels. Drivers are way less of a concern cause all you need is a clock, a network driver and a disk driver, not 30 different USB drivers, half a dozen networking drivers and more. Half of linux is just drivers cause it's designed to run on real machines whereas unikernels are explicitly designed to run as vms. Happy to engage with real life examples on these concerns but the best way to truly understand since there's all sorts of misunderstanding here is to just simply deploy one.
sp332 7y ago
For every single ? in your comment, the answer is "it's done at the hypervisor level". A unikernel is basically a single process, so privilege separation, users, groups etc are irrelevant. Drivers are much, much simpler (and I would argue, easier to audit) because they mostly only need to access virtualized, hypervisor-provided HAL APIs. Filesystem, DB, LDAP would be external - possibly provided by another VM on the same physical hardware.

mathattack 7y ago

Will they find a way to slip in the Watson brand? IBM is no longer in the innovation business. They (along with Red Hat) are in the audit business.

deleted 7y ago

[deleted]

zackmorris 7y ago

Quick question/idea about security: with all of the exploits today like Meltdown and Spectre, is anyone working on a truly secure container? Something like a public website running on an Intel processor, and/or maybe subdomains running on AMD and other mainstream CPUs. They would each be running some number of containers (up to say 1000 to start) and anyone could log in anonymously and run whatever they want in a minimal OS like Ubuntu. They would probably just have access to a small filesystem and maybe some open ports for HTTP or SSL or whatever. The challenge would be to break out and take over the host computer. If anyone does that, then the pwned container is flagged as not secure. The idea here is that until this happens, we can't really run a container and allow public access. I'd like containers to be as ubiquitous as websites running Javascript in the browser's sandbox so that we can host our own REPLs. If companies are serious about this, they should also consider providing bounties, say $100,000 guaranteeing that their container is secure for users. If you get hacked, you get the money. Without something like that, I'm afraid that I can't believe that containers are secure on faith alone.

lossolo 7y ago
This doesn't work like that. Containers (like docker, LXC and other based on cgroups etc) are isolated by the kernel but they share the same kernel. This means that kernel exploit will make all those containers insecure by your definition. If you go further and try to embed containers in hardware virtualization layer (so basically mini VMs) you still don't have any guarantees because from time to time new exploit emerges that allows to break out from guest OS to hypervisor. Then you have hardware layer and exploits based on speculative execution (Spectre, Meltdown and friends). Truly secure container doesn't exist. You would need some kind of mathematical proof for whole hardware stack and all communications between different hardware subsystems + whole kernel + container runtime.
- zackmorris 7y ago
  Hmmm that doesn't inspire a lot of confidence. I'd maybe be fine with a VM then, if containers fundamentally can't be made secure. Short of a mathematical "proof" of security, the best I could come up with was to leave containers open for the hackers of the world to exploit. If a container survives, that's a fairly strong guarantee that it can't be escaped (for now). What I am getting at is: if we know that hackers can break out of containers, then how are we ok with shared hosting running in containers? Once one gets owned, it's trivial to break into the others. Edit: this might be a way to force maintainers to fix kernel exploits as well.