7 ms·

You know the author of this piece is Brendan Gregg, right? DTrace author, performance guru, ex-Sun engineer and lead performance engineer at Joyent. And now a f

by Erwin 9y ago

You know the author of this piece is Brendan Gregg, right? DTrace author, performance guru, ex-Sun engineer and lead performance engineer at Joyent. And now a full time Linux guy. He doesn't think it's so bad apparently!

Annatar 9y ago
Yes, I know that; we even corresponded privately. However, call to authority does not magically validate Amazon, nor does it magically invalidate my points. Nothing Brendan wrote contradicts the points either of us made.
- openasocket 9y ago
  What is your point, exactly? That we shouldn't care about hardware-backed virtualization because we have zones? That AWS should go home and re-write EC2 to just be zones instead of VMs?
  - Annatar 9y ago
    What is your point, exactly? That we shouldn't care about hardware-backed virtualization because we have zones? Yes! That AWS should go home and re-write EC2 to just be zones instead of VMs? There is no reason for AWS to exist.
    - openasocket 9y ago
      Zones aren't a replacement for VMs. With VMs you can have different OSes, for example, which is not generally possible with a zone (you can do some stuff with syscall routing to do this in certain ways, but requires support from the root container). In a zone you can't do a variety of things that require higher level kernel access, like installing custom kernel modules. Zones also have a higher attack surface, which matters quite a lot when you are dealing with a multi-tenancy situation. But I'm pretty sure you knew that already.
      - Annatar 9y ago
        If you need another OS besides SmartOS for serving data, you have far bigger problems. Kernel modules are only meant to be installed in the global zone, on the hardware. This is by design. If you need access to that particular device, it can be delegated to that zone with the add subcommand to zonecfg(1M). Zones do not increase attack surface, I don't know what would make you think that. They reduce the attack surface because one can set up read-only zones, good luck changing anything in there. In a zone, attacker lands in a jail, assuming they actually manage to break in! What strange times we live in when there is so much misconception. So much knowledge lost! No wonder AWS and Linux rule the day! Zones are a revolutionary technology in terms of network security. What makes you think zones increase attack surface?
        openasocket 9y ago
        Zones increase attack surface compared to a hypervisor because you have a shared kernel, and every syscall is a potential attack surface. Compare that to a hypervisor, which a much simpler API exposed with far less functionality. Hypervisor vulnerabilities are far less common than kernel vulnerabilities for that reason. This is fairly well known in the security space. In fact, if you look into Joyent, you'll see they don't use regular zones for multi-tenancy, but rather a KVM-inspired hypervisor implementation of zones.
        Annatar 9y ago
        This is cookamamie. A hypervisor also has syscalls and in this particular case the global zone is the hypervisor. They even call it "type 1 hypervisor". Apropos "KVM-inspired hypervisor": the zones still function in exactly the same way as they did when I first used them in 2006; in fact when running lx and KVM Joyent is exploiting an obscure capability called a brand (branded zones), but that capability has always been there, since day one back in 2006. Finally, there has only been one known case of managing to break out of a zone and this was through a kernel driver exposed to it, exactly the kind of scenario you'd have on VMware. In the decade since, there have been no known cases, so not only is your argumentation cookamamie, it's in the realm of pure hypothesis.
- rjzzleep 9y ago
  I for one agree with you, but the main issue with SmartOS, Opensolaris, Solaris etc. is not the operating system. It's that some things are half baked and enterprisy. What do I mean by that? Brendan Gregg did mention in his article about Solaris migration certain things like amount of developers out of box performance etc. Here are a few issues that the Solaris crowd failed to address and if they don't address it soon imho they will fortify their own demise: Hardware availability, if you have a blade center with 10G intel hardware everything will work fine, but want to run it on a ryzen? tough luck. This already disqualifies a bunch of curious developers. KVM support? Sure, but it's not well documented the tooling if you don't use the enterprise management tools by Joyent is horrible. The standard tooling `vmadm` in SmartOS will leave logadm residues that will spam your notifications it's been an unfixed bug for multiple years. UEFI? Nope. The tooling and documentation is the worst. Want to migrate from libvirt? You have to shut it down, dd the image to the zfs zone and hope that it works. If it didn't well you can't access vnc before the host is already booted into a failed state. The actual killer: Documentation. The SmartOS confluence wiki starts off by giving you a taste and the typical enterprise way they offer you training. That works just fine so long as you're one of select few choices to pick from(Jira loves that model). But when you fail to attract the crowd of developers, you're bound to fall behind. There's exactly 3 books on OpenSolaris, 0 on SmartOS and a bunch of technical old documents on Solaris from Oracle. There's probably been at least 15 Docker books this year alone. Whole bunch to manage kubernetes. For creating network topologies in linux you pick up a bunch of books that aren't outdated. I love the idea of SmartOS and I'll run my machines as long as I feel it makes sense and you might argue that the complexity is a lot lower than what linux provides, but that's besides the point. What the point in crossbow, HVM, etc. when the only place of information is: a) an old oracle tech paper b) an old oracle/sun/joyent consultant c) one of 5 blog posts that explain how to run it in your home NAS d) a wiki with so basic information that it's guaranteed not to work on your system At least the complexity of Linux is visible, and you can find a solution somewhere. Rather than having to find a consultant that guards that solution with his life.
  - Annatar 9y ago
    The standard tooling `vmadm` in SmartOS will leave logadm residues that will spam your notifications it's been an unfixed bug for multiple years. So let's get this straight: when Linux zealots 15-20 years ago answered "you have the source! Fix it yourself!" that was okay, but when it comes to SmartOS, it's out of the question? Sure, but it's not well documented the tooling Read. The. Manual. Pages. It's a real UNIX and that means high quality documentation! No "Stack Overflow" vomit! The actual killer: Documentation. Repeat after me: I will read the manual pages. There was just a discussion on this subject recently here. b) an old oracle/sun/joyent consultant so old Solaris guys are no good? At least the complexity of Linux is visible, and you can find a solution somewhere. Linux has incredible kludges and brokenness under the hood, it's just that one needs to be an old UNIX guy to understand just how much of garbage "solutions" places like "Stack Overflow" offer. Rather than having to find a consultant that guards that solution with his life. I've always been happy to teach all that I know about UNIX and never had a complaint or heard an unkind word about my teaching efforts, on the contrary. Find yourself a mentor. "Stack Overflow" isn't one and could never substitute one.

by Erwin 9y ago

Annatar 9y ago

Yes, I know that; we even corresponded privately. However, call to authority does not magically validate Amazon, nor does it magically invalidate my points. Nothing Brendan wrote contradicts the points either of us made.

openasocket 9y ago
What is your point, exactly? That we shouldn't care about hardware-backed virtualization because we have zones? That AWS should go home and re-write EC2 to just be zones instead of VMs?
- Annatar 9y ago
  What is your point, exactly? That we shouldn't care about hardware-backed virtualization because we have zones? Yes! That AWS should go home and re-write EC2 to just be zones instead of VMs? There is no reason for AWS to exist.
  - openasocket 9y ago
    Zones aren't a replacement for VMs. With VMs you can have different OSes, for example, which is not generally possible with a zone (you can do some stuff with syscall routing to do this in certain ways, but requires support from the root container). In a zone you can't do a variety of things that require higher level kernel access, like installing custom kernel modules. Zones also have a higher attack surface, which matters quite a lot when you are dealing with a multi-tenancy situation. But I'm pretty sure you knew that already.
    - Annatar 9y ago
      If you need another OS besides SmartOS for serving data, you have far bigger problems. Kernel modules are only meant to be installed in the global zone, on the hardware. This is by design. If you need access to that particular device, it can be delegated to that zone with the add subcommand to zonecfg(1M). Zones do not increase attack surface, I don't know what would make you think that. They reduce the attack surface because one can set up read-only zones, good luck changing anything in there. In a zone, attacker lands in a jail, assuming they actually manage to break in! What strange times we live in when there is so much misconception. So much knowledge lost! No wonder AWS and Linux rule the day! Zones are a revolutionary technology in terms of network security. What makes you think zones increase attack surface?
      - openasocket 9y ago
        Zones increase attack surface compared to a hypervisor because you have a shared kernel, and every syscall is a potential attack surface. Compare that to a hypervisor, which a much simpler API exposed with far less functionality. Hypervisor vulnerabilities are far less common than kernel vulnerabilities for that reason. This is fairly well known in the security space. In fact, if you look into Joyent, you'll see they don't use regular zones for multi-tenancy, but rather a KVM-inspired hypervisor implementation of zones.
        Annatar 9y ago
        This is cookamamie. A hypervisor also has syscalls and in this particular case the global zone is the hypervisor. They even call it "type 1 hypervisor". Apropos "KVM-inspired hypervisor": the zones still function in exactly the same way as they did when I first used them in 2006; in fact when running lx and KVM Joyent is exploiting an obscure capability called a brand (branded zones), but that capability has always been there, since day one back in 2006. Finally, there has only been one known case of managing to break out of a zone and this was through a kernel driver exposed to it, exactly the kind of scenario you'd have on VMware. In the decade since, there have been no known cases, so not only is your argumentation cookamamie, it's in the realm of pure hypothesis.
rjzzleep 9y ago
I for one agree with you, but the main issue with SmartOS, Opensolaris, Solaris etc. is not the operating system. It's that some things are half baked and enterprisy. What do I mean by that? Brendan Gregg did mention in his article about Solaris migration certain things like amount of developers out of box performance etc. Here are a few issues that the Solaris crowd failed to address and if they don't address it soon imho they will fortify their own demise: Hardware availability, if you have a blade center with 10G intel hardware everything will work fine, but want to run it on a ryzen? tough luck. This already disqualifies a bunch of curious developers. KVM support? Sure, but it's not well documented the tooling if you don't use the enterprise management tools by Joyent is horrible. The standard tooling `vmadm` in SmartOS will leave logadm residues that will spam your notifications it's been an unfixed bug for multiple years. UEFI? Nope. The tooling and documentation is the worst. Want to migrate from libvirt? You have to shut it down, dd the image to the zfs zone and hope that it works. If it didn't well you can't access vnc before the host is already booted into a failed state. The actual killer: Documentation. The SmartOS confluence wiki starts off by giving you a taste and the typical enterprise way they offer you training. That works just fine so long as you're one of select few choices to pick from(Jira loves that model). But when you fail to attract the crowd of developers, you're bound to fall behind. There's exactly 3 books on OpenSolaris, 0 on SmartOS and a bunch of technical old documents on Solaris from Oracle. There's probably been at least 15 Docker books this year alone. Whole bunch to manage kubernetes. For creating network topologies in linux you pick up a bunch of books that aren't outdated. I love the idea of SmartOS and I'll run my machines as long as I feel it makes sense and you might argue that the complexity is a lot lower than what linux provides, but that's besides the point. What the point in crossbow, HVM, etc. when the only place of information is: a) an old oracle tech paper b) an old oracle/sun/joyent consultant c) one of 5 blog posts that explain how to run it in your home NAS d) a wiki with so basic information that it's guaranteed not to work on your system At least the complexity of Linux is visible, and you can find a solution somewhere. Rather than having to find a consultant that guards that solution with his life.
- Annatar 9y ago
  The standard tooling `vmadm` in SmartOS will leave logadm residues that will spam your notifications it's been an unfixed bug for multiple years. So let's get this straight: when Linux zealots 15-20 years ago answered "you have the source! Fix it yourself!" that was okay, but when it comes to SmartOS, it's out of the question? Sure, but it's not well documented the tooling Read. The. Manual. Pages. It's a real UNIX and that means high quality documentation! No "Stack Overflow" vomit! The actual killer: Documentation. Repeat after me: I will read the manual pages. There was just a discussion on this subject recently here. b) an old oracle/sun/joyent consultant so old Solaris guys are no good? At least the complexity of Linux is visible, and you can find a solution somewhere. Linux has incredible kludges and brokenness under the hood, it's just that one needs to be an old UNIX guy to understand just how much of garbage "solutions" places like "Stack Overflow" offer. Rather than having to find a consultant that guards that solution with his life. I've always been happy to teach all that I know about UNIX and never had a complaint or heard an unkind word about my teaching efforts, on the contrary. Find yourself a mentor. "Stack Overflow" isn't one and could never substitute one.