4 ms·

In this case, do clouds with physical bare-metal provisioning capabilities like EC2 / GCP / Azure / OCI kill the Joyent value proposition?

jen20 7y ago
Not really - since the bare metal provisioning tends only to be large, expensive instance types. The benefit of secure OS-level virtualisation is that you can co-locate the smaller workloads of multiple tenants on the same bare metal, while not incurring the overhead of hardware virtualisation. When you compared the utility of JCP "tiny" instance (1 CPU worth of shares, 256MB of RAM) to a t2.micro or similar instance in AWS, the difference was astounding. Part of this is the CPU credit mechanism on T2 instances, but I'd wager that a good part of it was being in OS-level virtualisation.
cookiecaper 7y ago
Docker killed the value proposition. "Worse is better". FreeBSD's jails and illumos's zones have always been far, far superior technically, providing actual kernel encapsulation and coherent administration utilities, but Docker had the triple combination of the familiarity of Linux, the cute whale, and the bonfire of VC dollars to drive adoption. There is still a great deal of administrative/technical value in using anything-but-Docker if you can tolerate the social stigma, but who can? The CxOs need that Docker/Kubernetes feather in their cap so they can look cool at conferences. Suggesting SmartOS/illumos gets you sideways looks at the very bare minimum. When I stated that I was thinking about using it here over a year ago, my inbox was immediately flooded with accusations that I was part of a Joyent-bankrolled cabal, because clearly no sane ordinary person would consider such a dead platform. In the interim, OmniTI has gone belly-up and OpenZFS has deemed ZFS on Linux the rightful heir to the ZFS throne, and now the Joyent Public Cloud is shuttering. If the impression that you had to be part of a conspiracy to use SmartOS for a greenfield project was reasonable then, it's only more reasonable now. The sun may yet rise again on illumos some day, but it's time to accept that it's well into the night now.
RantyDave 7y ago
Joyent's value proposition was killed (for the most part) by the experience of using their public interface. It would've taken a great deal of bravery to try that and decide a local install would be better. The node thing also did a lot of damage - Joyent wrote a lot of the SmartOS/Triton command line tools in node so they were slow as hell. Triton itself is a very non-trivial install although quite probably less so than a complete k8s rig. There are also some really subtle Linux compatibility problems. For instance: IIRC a SmartOS malloc will not fail. Ever. But it might not return either. There's a really complicated mark/sweep type virtual memory thing and the documentation is good, for the nasty node stuff, and kinda terrible otherwise. Finally, and with no lack of irony, you just can't run SmartOS on the cloud. For a start it needs to do its first boot off a CD and subsequent boots off a USB drive-type-arrangement. The hard part is that most cloud providers run a butchered Linux kernel and not a fully generalised VM - and on vultr at least, while they run a proper VM, for some reason the processors don't "come up". I've not looked at all this stuff for at least a year so it may have changed. Either which way I believe the majority of the community is now on OmniOS - which I always got on better with anyway.
- jen20 7y ago
  > Triton itself is a very non-trivial install Actually I'd say the Triton install is not only trivial, but should be a model for other similar systems. During my testing with Intel NUCs, I had a usable system deploying containers in under 20 minutes. Most managed Kubernetes systems cannot even provision in that time frame. > you just can't run SmartOS on the cloud. The installer may lead you to the USB and install path, but it is not necessary. I built Vagrant boxes that had SmartOS on a regular disk, and I believe that more recently SmartOS has been running well in GCP. There is no reason it could not run in AWS or Azure, either.
  - merb 7y ago
    > Most managed Kubernetes systems cannot even provision in that time frame. when was the last time you used k8s?! I mean as of now there is even the expirmental k3os which boots a fully fledged k8s in seconds. you can even built a cluster with it. sadly HA backplane is still not available since k3s is still early in its development.

jen20 7y ago

Not really - since the bare metal provisioning tends only to be large, expensive instance types. The benefit of secure OS-level virtualisation is that you can co-locate the smaller workloads of multiple tenants on the same bare metal, while not incurring the overhead of hardware virtualisation. When you compared the utility of JCP "tiny" instance (1 CPU worth of shares, 256MB of RAM) to a t2.micro or similar instance in AWS, the difference was astounding. Part of this is the CPU credit mechanism on T2 instances, but I'd wager that a good part of it was being in OS-level virtualisation.

cookiecaper 7y ago

Docker killed the value proposition. "Worse is better". FreeBSD's jails and illumos's zones have always been far, far superior technically, providing actual kernel encapsulation and coherent administration utilities, but Docker had the triple combination of the familiarity of Linux, the cute whale, and the bonfire of VC dollars to drive adoption. There is still a great deal of administrative/technical value in using anything-but-Docker if you can tolerate the social stigma, but who can? The CxOs need that Docker/Kubernetes feather in their cap so they can look cool at conferences. Suggesting SmartOS/illumos gets you sideways looks at the very bare minimum. When I stated that I was thinking about using it here over a year ago, my inbox was immediately flooded with accusations that I was part of a Joyent-bankrolled cabal, because clearly no sane ordinary person would consider such a dead platform. In the interim, OmniTI has gone belly-up and OpenZFS has deemed ZFS on Linux the rightful heir to the ZFS throne, and now the Joyent Public Cloud is shuttering. If the impression that you had to be part of a conspiracy to use SmartOS for a greenfield project was reasonable then, it's only more reasonable now. The sun may yet rise again on illumos some day, but it's time to accept that it's well into the night now.

RantyDave 7y ago

Joyent's value proposition was killed (for the most part) by the experience of using their public interface. It would've taken a great deal of bravery to try that and decide a local install would be better. The node thing also did a lot of damage - Joyent wrote a lot of the SmartOS/Triton command line tools in node so they were slow as hell. Triton itself is a very non-trivial install although quite probably less so than a complete k8s rig. There are also some really subtle Linux compatibility problems. For instance: IIRC a SmartOS malloc will not fail. Ever. But it might not return either. There's a really complicated mark/sweep type virtual memory thing and the documentation is good, for the nasty node stuff, and kinda terrible otherwise. Finally, and with no lack of irony, you just can't run SmartOS on the cloud. For a start it needs to do its first boot off a CD and subsequent boots off a USB drive-type-arrangement. The hard part is that most cloud providers run a butchered Linux kernel and not a fully generalised VM - and on vultr at least, while they run a proper VM, for some reason the processors don't "come up". I've not looked at all this stuff for at least a year so it may have changed. Either which way I believe the majority of the community is now on OmniOS - which I always got on better with anyway.

jen20 7y ago
> Triton itself is a very non-trivial install Actually I'd say the Triton install is not only trivial, but should be a model for other similar systems. During my testing with Intel NUCs, I had a usable system deploying containers in under 20 minutes. Most managed Kubernetes systems cannot even provision in that time frame. > you just can't run SmartOS on the cloud. The installer may lead you to the USB and install path, but it is not necessary. I built Vagrant boxes that had SmartOS on a regular disk, and I believe that more recently SmartOS has been running well in GCP. There is no reason it could not run in AWS or Azure, either.
- merb 7y ago
  > Most managed Kubernetes systems cannot even provision in that time frame. when was the last time you used k8s?! I mean as of now there is even the expirmental k3os which boots a fully fledged k8s in seconds. you can even built a cluster with it. sadly HA backplane is still not available since k3s is still early in its development.