5 ms·

I've been running SmartOS containers for 6+ years and it's the best of all worlds. Easy service management, works with regular binaries or Dockerized deploys,

I've been running SmartOS containers for 6+ years and it's the best of all worlds. Easy service management, works with regular binaries or Dockerized deploys, SDN always built-in, service discovery built-in (CNS), even Windows VMs available. Best kept secret in the cloud even though the entire stack is open-source. On the orchestration side I use Puppet Pipelines (formally Distelli) which works with standard VMs, SmartOS containers, Docker, Linux images, from packaging through testing and deployment. And they just lowered pricing (!??!) There's no reason to be locked into Kubernetes unless you're sure you need it. SmartOS scales in seconds. As CTO I'm always checking out the next platform, but all the newer solutions look much more complicated. EDIT: Heroku has weird errors when you push CPU or RAM, also the CPUs aren't so fast, Linode can't be trusted, Digital Ocean is OK but is still very manual roll-your-own, AWS is a behemoth and medium fast, Google is fine but specialized and hard to migrate away from, OpenStack isn't for small orgs... I've run production services on them all. It's easy to try a new service with a Pipelines-style orchestration because it doesn't care which service its talking to. And makes it obvious when platform-specific instructions or concessions are required.

sansnomme 8y ago
This looks interesting, I have a few questions. How do you deal with Linux dependencies etc.? All your applications must be BSD compatible? Any advice for getting started with it on DigitalOcean? Or is nested out of the story?
- doublerebel 8y ago
  Oh, SmartOS containers run Linux seamlessly too. I guess I take that knowledge for granted now. But many many of the popular cloud tools (hashicorp consul and vault, redis, couchdb, arangodb, postgres) run natively on SmartOS and have better resource and user management from the underlying system. I haven't used Digital Ocean's kube offering. Otherwise it's much like Linode or other services from that era -- install your own distro and binaries, which needs some kind of orchestration automation to scale.
  - sansnomme 8y ago
    Do you think you can do a blog post or tutorial for running SmartOS on the popular providers and getting to the point of deploying a basic Rails app with PostgreSQL in Docker etc.? The SmartOS documentation looks ...sparse, and there aren't many recent blog posts on SmartOS, most existing ones are from <2017.
    - doublerebel 8y ago
      SmartOS runs on Hetzner, OVH, EveryCity, MNX, GigaVPS, and more. The nice thing about Solaris: the system doesn't change too rapidly, so the documentation doesn't need to change either. On a SmartOS container, Rails and Postgres can be installed from the package manager and automatically configured as a service. Running a Docker image on Joyent Triton cloud is just a one-line CLI command. SmartOS is built for the cloud, and is usually semi-managed by the provider. If it's worth your effort to run a separate SmartOS cluster, you'd know. But there is also Project FIFO for that: https://project-fifo.net https://project-fifo.net
  - weavie 8y ago
    Common Lisp does not run in LX zones.. at least Clozure common lisp doesn't. Apparently Clozure does something 'ungodly' to the signal stack which just throws the kernel and ends up crashing out. Chances are, these days anyway, most people aren't running common lisp, so it probably isn't much of a problem - plus Clozure runs perfectly fine under Solaris. As a result it's probably not high on anyones list to sort out.
    - jclulow 8y ago
      It was true that LX didn't initially handle what Clozure does with signals, but I recall adding a mechanism to work around the busted behaviour and I'd expect it to work now. Certainly in some basic testing of CCL things appeared dramatically improved, though this was some time ago now.
      - weavie 8y ago
winrid 8y ago
Thanks for your insight. Why can't Linode be trusted?
- bifrost 8y ago
  They've proven many times that they're not. Read through slashdot and google for "linode bitcoin stolen"
cyberpunk 8y ago
I was also on SmartOS for a long time; however with what happened to most of us old timers when sun went down in the back of my mind, and with the way things are looking in illumos land -- we decided finally it's not a good bet for us to be dependent on a single company for our OS anymore. While I'm sure Samsung will keep Joyent/SmartOS kicking for now, I'm uneasy being dependent on them. We've since started a migration over to FreeBSD for our infra hosts which do all the same things (mostly running linux and bsd guests, zfs tricks and so on) and the experience has been largely positive. No numbers to back it up, but I've observed performance is a bit better and having more options than ipfilter in the base system is very welcome. YMMV of course, but you may want to consider an extra basket for all those eggs!

sansnomme 8y ago

This looks interesting, I have a few questions. How do you deal with Linux dependencies etc.? All your applications must be BSD compatible? Any advice for getting started with it on DigitalOcean? Or is nested out of the story?

doublerebel 8y ago
Oh, SmartOS containers run Linux seamlessly too. I guess I take that knowledge for granted now. But many many of the popular cloud tools (hashicorp consul and vault, redis, couchdb, arangodb, postgres) run natively on SmartOS and have better resource and user management from the underlying system. I haven't used Digital Ocean's kube offering. Otherwise it's much like Linode or other services from that era -- install your own distro and binaries, which needs some kind of orchestration automation to scale.
- sansnomme 8y ago
  Do you think you can do a blog post or tutorial for running SmartOS on the popular providers and getting to the point of deploying a basic Rails app with PostgreSQL in Docker etc.? The SmartOS documentation looks ...sparse, and there aren't many recent blog posts on SmartOS, most existing ones are from <2017.
  - doublerebel 8y ago
    SmartOS runs on Hetzner, OVH, EveryCity, MNX, GigaVPS, and more. The nice thing about Solaris: the system doesn't change too rapidly, so the documentation doesn't need to change either. On a SmartOS container, Rails and Postgres can be installed from the package manager and automatically configured as a service. Running a Docker image on Joyent Triton cloud is just a one-line CLI command. SmartOS is built for the cloud, and is usually semi-managed by the provider. If it's worth your effort to run a separate SmartOS cluster, you'd know. But there is also Project FIFO for that: https://project-fifo.net https://project-fifo.net
- weavie 8y ago
  Common Lisp does not run in LX zones.. at least Clozure common lisp doesn't. Apparently Clozure does something 'ungodly' to the signal stack which just throws the kernel and ends up crashing out. Chances are, these days anyway, most people aren't running common lisp, so it probably isn't much of a problem - plus Clozure runs perfectly fine under Solaris. As a result it's probably not high on anyones list to sort out.
  - jclulow 8y ago
    It was true that LX didn't initially handle what Clozure does with signals, but I recall adding a mechanism to work around the busted behaviour and I'd expect it to work now. Certainly in some basic testing of CCL things appeared dramatically improved, though this was some time ago now.
    - weavie 8y ago

winrid 8y ago

Thanks for your insight. Why can't Linode be trusted?

bifrost 8y ago
They've proven many times that they're not. Read through slashdot and google for "linode bitcoin stolen"

cyberpunk 8y ago

I was also on SmartOS for a long time; however with what happened to most of us old timers when sun went down in the back of my mind, and with the way things are looking in illumos land -- we decided finally it's not a good bet for us to be dependent on a single company for our OS anymore. While I'm sure Samsung will keep Joyent/SmartOS kicking for now, I'm uneasy being dependent on them. We've since started a migration over to FreeBSD for our infra hosts which do all the same things (mostly running linux and bsd guests, zfs tricks and so on) and the experience has been largely positive. No numbers to back it up, but I've observed performance is a bit better and having more options than ipfilter in the base system is very welcome. YMMV of course, but you may want to consider an extra basket for all those eggs!