10 ms·

Triton: Docker and the “best of all worlds”

202 points by thepumpkin1979 12y ago

kordless 12y ago
Here's the signup link for Joyent's hosted container service: https://www.joyent.com/lp/preview https://www.joyent.com/lp/preview. We're seeing the initial explosion of the microservices ecosystem right now. I've been spending most of my time over the last few years thinking about trusted decentralized infrastructure and have decided that microservices, including hosted ones, could be one possible solution for instantiating trust as a proper 'knob' of the cloud. This Intracloud, if I'm allowed to use a new buzz term, will be smeared across 100s (1000s?) of datacenters world-wide. Have a high-trust use-case? Run it in a German or Dutch datacenter. Have a high-efficiency use case? Run it on a friend's cluster for free. I believe in this idea of the 'trust knob' enough that I sought out a job with the folks at https://giantswarm.io/ https://giantswarm.io/, where I am now a dev evangelist. We are a German hosted microservices stack which provides a Docker platform. Alpha signup is here: https://giantswarm.io/request-invite/ https://giantswarm.io/request-invite/. Demo of it in action here: https://github.com/kordless/swarm-ngrok#ngrokn-giant-swarm https://github.com/kordless/swarm-ngrok#ngrokn-giant-swarm I also hacked together a SF Microservices meetup last Friday: http://www.meetup.com/SF-Microservices/ http://www.meetup.com/SF-Microservices/. 139 people have already joined. Would like some feedback for content! Planning on mid-April for the first event. I'm excited to see what happens with this market in the next 6 months!
SandB0x 12y ago
Can someone explain this in words that a time traveller from 2010 would understand?
- simtel20 12y ago
  Well, since 2010 Sun has died, and opensolaris is no more. It lives on as an open source effort called Illumos. At one point, solaris could run Linux binaries unmodified under a solaris kernel and userspace. That became un-maintained, but it is now working again. One of the reasons this feature fell out of use was because it could only run 32-bit linux binaries. Now it can run 64-bit binaries too. If you want to try it out there is a company you may not be familiar with, dear time-traveler from 2010, called Joyent, that will let you use this. What is very interesting dear time traveler is that there is a format for containing programs and their environments called docker that allows you to use features, as you could in a zone, but which is easier to use than zones. You can deploy your containers built on linux to solaris zones with this feature, and Joyent lets you do it.
  - pyre 12y ago
    Oracle bought Sun in 2009 and finalized in Jan. 2010, so this time traveller would have to be from a very specific slice of 2010 to not know about the demise of Sun. :P
- deleted 12y ago
  [deleted]
- kordless 12y ago
  Remember when virtual machines came out? That technology gave us tools to create a 'layered' effect for compute related resources on a single, managed machine. Microservice technologies, like Docker containers, are a logical extension of that layering into the process space on the machine, and in many cases multiple virtual machines running on a single machine. It's all about the layers, man.
  - steeef 12y ago
    Joyent actually extends this and allows you to treat an entire datacenter as your docker host, similar to how you can use a product like VMware to cluster ESX hosts together.
- mrfusion 12y ago
  Love the question. This should be the new ELI5. I think a lot of articles on here would benefit from answering that question.
  - evanspa 12y ago
    Based on your username, I'm not surprised you like the time-traveling angle.
  - tomcam 12y ago
    Ditto. It was a boon to me too.
  - raus22 12y ago
    The Explain it like i have been in a coma for 5 years... Elihbiacf5y or just 5 years in coma, explain! 5YICE.... Or ETTT5YA or ET5... explain to time traveler from 5 years ago
- pgeorgi 12y ago
  I assume that the time traveller knows Open Solaris up to 2010. Someone used Linux facilities that are somewhat similar to Solaris Zones and ZFS snapshots (you'll know those, since Solaris 10 from 2005 shipped with that and wasn't even the first) to build an ecosystem for lightweight virtual systems with standardized interfaces. That system is called Docker (and runs on Linux, obviously) OpenSolaris is all but dead, but a community succeeded it, called IllumOS. People from that community decided to attempt to use Solaris Zones, with Linux branded zones (that capability was dropped in the meantime so they had to invest some effort to bring it up again) to reimplement Docker on an IllumOS (ie. OpenSolaris) foundation.
jimpick 12y ago
Nice!
thinkingkong 12y ago
Tldr. Triton lets you run your docker containers in the cloud without worrying about virtual machines or setting up your own paas. Signup here: https://www.joyent.com/lp/preview https://www.joyent.com/lp/preview
- hueving 12y ago
  So how is this different from the container service from google?
  - thinkingkong 12y ago
    Frankly, the technical implementation.
  - brianshaler 12y ago
    One interesting bit from the article: "Does this mean that I'll be able to DTrace my Linux apps in a Docker container?" (yes) Technically you can run dtrace on various platforms, and I don't know enough to tell you specific reasons why you would rather use dtrace on smartos than linux. I got the impression a few years ago the ports weren't as good. (perhaps relating to the linux port not yet being a kernel module?)
    - lloydde 12y ago
      Debugging LX branded zones on SmartOS https://www.youtube.com/watch?v=6oIBiWdh41c https://www.youtube.com/watch?v=6oIBiWdh41c "A gritty how-to on debugging LX branded zones on SmartOS presented internally at Joyent on 1/30/15. While given to an internal audience, there is quite a bit of information here that may be interesting to the broader community"
- mcescalante 12y ago
  Thanks, I spent a minute skimming through this post and ended up coming here hoping someone wrote a tl;dr like this. Must be challenging to write effective copy for something like this (not being sarcastic)
- 12y ago
phildougherty 12y ago
"You can run your Docker containers across entire data centers without ever creating a "cluster" as other IaaS providers would have you do." This is a little unclear. I get that you don't have to create a "cluster" because you're going to use the API to launch containers on existing dedicated hardware, but how is that a WIN for your finance team? I guess I'm missing something.. but wouldn't you have to already have systems online that you're paying for that these containers can be launched on? From: https://www.joyent.com/blog/docker-bake-off-aws-vs-joyent https://www.joyent.com/blog/docker-bake-off-aws-vs-joyent
- thinkingkong 12y ago
  The difference for the operator is: Baremetal -> smartos -> container vs Baremetal -> hypervisor -> virtual machine -> container. Theres root safety in the smartos implementation of docker. So they can do multi-tenancy. For the customer, you dont have to provision entire virtual machines to run docker containers.
  - phildougherty 12y ago
    Is it multi-tenancy of various Joyent customers on the same hardware? So you get containers placed somewhere in their data center on shared hardware with other customers? Or do you already need to have dedicated hardware provisioned at Joyent to launch containers on? I'm still unclear on that point, which makes me question how it's saving the end user money.
    - xj9 12y ago
      Yes, you are running on shared hardware, securely, but with no hypervisor. The biggest money-saver is in performance. IO-heavy applications can see anywhere from 5-10x performance improvements[1] by switching from hardware- to OS-virtualization. [1]: http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-zones-kvm-xen/ http://dtrace.org/blogs/brendan/2013/01/11/virtualization-pe...
    - misterbisson 12y ago
      Yes, the the containers are running on bare-metal multi tenant hardware. Some of the savings comes from the performance gains as noted elsewhere, but additional savings comes from not having to pay for and manage an additional layer of virtual machines. There's no need to provision anything other than the containers.
redwood631 12y ago
This is running Linux binaries on Oracle Solaris (which they call SmartOS), and not the real thing that Oracle is still developing, but a fork from 2010 that is maintained by 1/10th of the developers it once had. https://www.openhub.net/p/illumos https://www.openhub.net/p/illumos. This would have been pretty rad 10 years ago, when the world still cared about Solaris. On a minor note: the post doesn't credit Oracle or Solaris, from where more than 90% of their SmartOS code comes from, until Oracle closed their code in 2010.
- thinkingkong 12y ago
  Thats because Oracle deserves no credit. The illumos project was created as a derivative of opensolaris which was created by Sun. The lineage is actually sort of mind boggling. If we want to be pedantic, we could complain about nobody is giving credit to BSD.
  - jimpick 12y ago
    And Bell Labs.
- pyre 12y ago
  > until Oracle closed their code in 2010 At that point, why credit Oracle instead of Sun? Oracle finalized their deal to buy Sun in January 2010. How much did Oracle contribute between then and the announcement that they were closing their code? Edit: Oracle announced closing OpenSolaris on August 13, 2010. The last "release" of OpenSolaris was June 2009.
  - bahamat 12y ago
    Oracle never publicly announced it. The last commit that I see copyrighted by Oracle is https://github.com/illumos/illumos-gate/commit/ea10ff14a02f7788b8ec27b4b21b925542a8cca6 https://github.com/illumos/illumos-gate/commit/ea10ff14a02f7... on August 18, 2010. After that, updates from Oracle silently stop. But your point stands well. Oracle only had a paltry 7-8 months of contributions. In my opinion, and I really don't know who would share this view, since Sun opened Solaris, illumos is now what Solaris was intended to be; it's not the illumos community that forked Solaris, it was Oracle.
josh2600 12y ago
Joyent has solved some really hard problems to make this work. I was lucky enough to get the chance to sit down with Bryan last week and talk with him about what this took. It's essentially reimplementing Linux on top of Illumnos, which is no tiny feat. There are still messy hurdles to running Docker in production, but it's clear that Joyent has really tried to make something awesome here. As someone who works deeply in the future-container space, I applaud folks who are taking us deeper down the rabbit hole. I think it's clear that this whole 'back-to-the-future' isolation technology is really cool stuff. Jails/zones/containers have been around forever and I think it's great that we're finally taking advantage of this technology. Edit: at Terminal.com, we want people to push Linux forward, and this is a great example of taking Linux to new and intriguing heights. I did not think we would have Linux on Illumnos in quite this way in 2015 and it's delightful to see. We are all standing on the shoulders of giants and it's great to reach new vistas.
- kordless 12y ago
  +1 on terminal.com. Love it.
- justincormack 12y ago
  Isn't it an example of taking Solaris to new and intriguing heights? There is no actual Linux code in the implementation.
  - josh2600 12y ago
    That's a great point, but I think the point of what Bryan has been doing is to make Linux work with Zones (and dtrace). That's a primitive that Joyent has wanted to upstream into the Linux kernel for a long time and has never been able to get the necessary consensus around it (similar to OpenVZ's troubles getting their work upstreamed). In short, this is sort of a hack to give you zones on Linux without needing to get zones into the upstream. Yes, there's no linux code, but there is a lot of required understanding of Linux code to make something like this work. It's kinda amazing that they got 64-bit linux to run on top of Illumnos, right? I did not see that coming and maybe that's because I'm ignorant in some capacity, but it's been a pleasant surprise.
mrinterweb 12y ago
Is there any pricing information for Triton yet?
- jholmes83 12y ago
  Pricing here: https://www.joyent.com/blog/expanded-container-service-preview https://www.joyent.com/blog/expanded-container-service-previ...
  - mrinterweb 12y ago
    Thanks for finding that link. That is a very attractive pricing model. I am currently running about 10 VPS running about 25 containers. The costs of the VPS hosting is affordable, but managing those VPS hosts, deployment, balancing, discovery, and scaling are the hard parts. I'm looking forward to learning more about Triton.
themgt 12y ago
Didn't see the actual repo linked: https://github.com/joyent/sdc-docker https://github.com/joyent/sdc-docker "A Docker Engine for SmartDataCenter, where the whole DC is exposed as a single docker host. The Docker remote API is served from a 'docker' core SDC zone (built from this repo). ... Disclaimer: This is still very much alpha. Use at your own risk!"
- trentmick 12y ago
  Thanks for the link. I've updated the "current state" notes to better reflect reality. https://github.com/joyent/sdc-docker#current-state https://github.com/joyent/sdc-docker#current-state
- ochoseis 12y ago
  To me it seems like Triton is about shoehorning Linux into Solaris/SmartOS containers. While this seems great for accelerating adoption, why not just use SmartOS-specific Dockerfiles/repos in conjunction with sdc-docker (Not directly addressing the parent, more of a general question)? Imagine all of the crazy bugs and edge cases that will arise when trying to run Linux binaries on UNIX. I'm surely making some assumptions, but the primary languages/developers that something like this targets are probably like Java, Javascript/Node.js, Ruby, Python, C...which are probably all portable enough to run on SmartOS without much/any modification.
  - wmf 12y ago
    Docker multiarch support isn't finished yet, and getting all the world's Dockerfiles rebuilt on SmartOS is a big task. There are also different philosophies about whether container images should be distributed as source (Dockerfiles) or as binaries. It looks like Joyent decided to run Linux/x86-64 binaries to be on the safe side.
crashoverdrive 12y ago
I feel like perhaps I'm the only one concerned about this: (Somehow, “SmartOS + LX-branded zones + SmartDataCenter + sdc-portolan + sdc-docker” was a bit of a mouthful.) Not only is it a mouthful, it means there is a much wider space for things to go wrong.
- tracker1 12y ago
  Compared to say "hypervisor + Linux + kubernetes + weave + etcd + lxc-docker" ...? Not to mention all the pieces that work together for lxc-docker.
23david 12y ago
Excited to see this. Congrats to the team on the release!
bau5 12y ago
I saw the name and my desire to read it plummeted. The same person wrote http://www.joyent.com/blog/the-power-of-a-pronoun http://www.joyent.com/blog/the-power-of-a-pronoun which led to one of the most talented developers of node core and libuv leaving the project for a while.
- rubiquity 12y ago
  What relevance does that have to this? It's sad that your post is even being upvoted.
- stuaxo 12y ago
  Wow, bloody hell - hadn't read that before.
  - cookiecaper 12y ago
    Me either. Wondering how I missed all that drama but not particularly sad I did.
- codygman 12y ago
  The power of a pronoun is a great article, if anything him writing it should be a plus. The only way it is a negative is if you feel gendered pronouns don't further sexism, but I don't see how you could support that argument.
  - crazy1van 12y ago
    Does Joyent refuse to hire speakers of Spanish, French, Italian, Hindi, or any of the other dozens of languages that have gender throughout the entire grammar?
    - nosequel 12y ago
      You are missing the entire point. It is one thing to not allow gender pronouns (what you are suggesting) versus punishing someone who specifically set out to change gender-neutral pronouns into male dominated ones (the Joyent case). REAL BIG difference.
DannoHung 12y ago
Uh, so I've never really heard of SmartOS or Joyent's SDC before now... This stuff looks pretty fucking unreal. What's the catch? It looks like it solves so many problems with clustered container deployment. Exposing a whole datacenter as a single Docker host seems like the end-game to me. Am I missing something? I'm a little low on sleep today. edit: To answer my own question a little: I guess the docker-sdc isn't quite fully baked yet. That's not really a huge issue since it's still a preview, I think?
- bananaoomarang 12y ago
  Nope, SmartOS is pretty neat.
  - ssmoot 12y ago
    Ditto. We were early adopters at work. KVM + ZFS and practically no setup is ace.
- jgrowl 12y ago
  I'm right there with you. It feels like the beginning of skynet
- pannon 12y ago
  No catch, it is a truly innovative/disruptive technology.
- _delirium 12y ago
  The main catch (which may also be a plus) is that the Joyent set of technologies is quite opinionated about how to manage a cloud, so you more or less have to "buy in" and do things its way. Your node OS will be SmartOS, nodes will be PXE-booted, your filesystem will be ZFS, the network topology will work in a specific way, etc. More practically, it also takes some time to be proficient in operating it (or even just get it working), because there are quite a few moving parts. Getting a SmartDataCenter deployment running, and then things like Manta and now sdc-docker running on top of it, is more than an afternoon's work. But if you're willing to buy in to the system and its choices fit your needs, it's really well engineered, imo much more "done right" than e.g. OpenStack is.
kstenerud 12y ago
Is there any further work going on to support AMD in the KVM driver? I remember the codebase having sat idle for a few years last time I checked...
- snw 12y ago
  While sadly not merged upstream yet, it works very well. I've been running custom SmartOS "eait" builds from http://imgapi.uqcloud.net/builds http://imgapi.uqcloud.net/builds (which include AMD support) for some time now without any issues. The source can be found at https://github.com/arekinath/smartos-live https://github.com/arekinath/smartos-live
themgt 12y ago
I find these to be pretty weasel words: Is Triton open source? Yes. Triton Elastic Container Infrastructure is the commercial offering built on the top of the open-source SmartDataCenter cloud infrastructure management platform. In addition to the open source components, Triton includes support, a DevOps portal, and intellectual property indemnification only available to commercial customers. In more plain English, "Yes. Triton is open source just like Mac OS X" https://www.joyent.com/developers/triton-faq#os https://www.joyent.com/developers/triton-faq#os
- lloydde 12y ago
  That's a good thing to clarify. It's not an open core system like Mac OS X. SmartDataCenter is open source and Triton is our product branding. https://github.com/joyent/sdc https://github.com/joyent/sdc There is work in progress to clean up and open source the self-serve portal and its dependencies. The propriety bits are not ours: "The Joyent SDC USB images offered to our customers contain certain firmware binaries for use with Joyent-branded hardware. We cannot make that software generally available in the open source builds; however, it is available to all who have purchased Joyent-branded hardware and is of no use without that hardware." https://docs.joyent.com/sdc7/obtaining-software https://docs.joyent.com/sdc7/obtaining-software
  - DannoHung 12y ago
    What does the bit about security patches and minor and major updates mean? The Product version includes automatic updates and the open source version you manage yourself? Or something else?
    - lloydde 12y ago
      Would you quote the bits here, so I can clarify with the product side of the house? We have not and never will let anything jeopardize the continued security of any SDC deployments. SDC having update channels is under active development: 'As of sdcadm version 1.5.5 and the release-20150319, SmartDataCenter has preliminary support for update "channels". A "channel" is separate stream of built SDC components with different stability characteristics.' https://github.com/joyent/sdc/blob/master/docs/operator-guide/update.md https://github.com/joyent/sdc/blob/master/docs/operator-guid... I image that the support channel will be the most conservative.
yonran 12y ago
Very cool. Is the next step integration with Manta so you can specify an environment of Linux executables to map over your data?
- dap 12y ago
  Yes, that's definitely the direction that we're headed. Manta allows users to spin up large numbers of transient containers around their data (i.e., without moving data around) in order to do map-reduce operations, and without having to think about the container management. Today, these compute tasks are specified as scripts that run inside a well-stocked SmartOS container, plus some optional assets to download (for custom binaries or other data). The obvious next step would be to let people specify they want to run an LX zone (rather than a SmartOS zone) so that they can run GNU/Linux binaries. Then we can consider whether it makes sense to phrase the task as a Docker image. That might well be a more natural way to incorporate a larger bunch of binaries and other shared data. Architecturally, all of this should be relatively straightforward (famous last words!).
batbomb 12y ago
> When Docker first rocketed I thought this was an interesting way to start off the post.
Bedon292 12y ago
This looks like a very nice way to deploy containers. I see the per hour prices (https://www.joyent.com/blog/expanded-container-service-preview https://www.joyent.com/blog/expanded-container-service-previ...) but is there any pricing on bandwidth? I imagine there will be eventually, right?
- adamtaro 12y ago
  I'd suggest looking at this page as an indicator (https://www.joyent.com/public-cloud/pricing https://www.joyent.com/public-cloud/pricing -- Bandwidth tab): all inbound traffic free, outbound is 12 cents/GB after the first GB, cheaper at higher tiers.

202 points by thepumpkin1979 12y ago

Here's the signup link for Joyent's hosted container service: https://www.joyent.com/lp/preview https://www.joyent.com/lp/preview. We're seeing the initial explosion of the microservices ecosystem right now. I've been spending most of my time over the last few years thinking about trusted decentralized infrastructure and have decided that microservices, including hosted ones, could be one possible solution for instantiating trust as a proper 'knob' of the cloud. This Intracloud, if I'm allowed to use a new buzz term, will be smeared across 100s (1000s?) of datacenters world-wide. Have a high-trust use-case? Run it in a German or Dutch datacenter. Have a high-efficiency use case? Run it on a friend's cluster for free. I believe in this idea of the 'trust knob' enough that I sought out a job with the folks at https://giantswarm.io/ https://giantswarm.io/, where I am now a dev evangelist. We are a German hosted microservices stack which provides a Docker platform. Alpha signup is here: https://giantswarm.io/request-invite/ https://giantswarm.io/request-invite/. Demo of it in action here: https://github.com/kordless/swarm-ngrok#ngrokn-giant-swarm https://github.com/kordless/swarm-ngrok#ngrokn-giant-swarm I also hacked together a SF Microservices meetup last Friday: http://www.meetup.com/SF-Microservices/ http://www.meetup.com/SF-Microservices/. 139 people have already joined. Would like some feedback for content! Planning on mid-April for the first event. I'm excited to see what happens with this market in the next 6 months!

SandB0x 12y ago

Can someone explain this in words that a time traveller from 2010 would understand?

simtel20 12y ago
Well, since 2010 Sun has died, and opensolaris is no more. It lives on as an open source effort called Illumos. At one point, solaris could run Linux binaries unmodified under a solaris kernel and userspace. That became un-maintained, but it is now working again. One of the reasons this feature fell out of use was because it could only run 32-bit linux binaries. Now it can run 64-bit binaries too. If you want to try it out there is a company you may not be familiar with, dear time-traveler from 2010, called Joyent, that will let you use this. What is very interesting dear time traveler is that there is a format for containing programs and their environments called docker that allows you to use features, as you could in a zone, but which is easier to use than zones. You can deploy your containers built on linux to solaris zones with this feature, and Joyent lets you do it.
- pyre 12y ago
  Oracle bought Sun in 2009 and finalized in Jan. 2010, so this time traveller would have to be from a very specific slice of 2010 to not know about the demise of Sun. :P
deleted 12y ago
[deleted]
kordless 12y ago
Remember when virtual machines came out? That technology gave us tools to create a 'layered' effect for compute related resources on a single, managed machine. Microservice technologies, like Docker containers, are a logical extension of that layering into the process space on the machine, and in many cases multiple virtual machines running on a single machine. It's all about the layers, man.
- steeef 12y ago
  Joyent actually extends this and allows you to treat an entire datacenter as your docker host, similar to how you can use a product like VMware to cluster ESX hosts together.
mrfusion 12y ago
Love the question. This should be the new ELI5. I think a lot of articles on here would benefit from answering that question.
- evanspa 12y ago
  Based on your username, I'm not surprised you like the time-traveling angle.
- tomcam 12y ago
  Ditto. It was a boon to me too.
- raus22 12y ago
  The Explain it like i have been in a coma for 5 years... Elihbiacf5y or just 5 years in coma, explain! 5YICE.... Or ETTT5YA or ET5... explain to time traveler from 5 years ago
pgeorgi 12y ago
I assume that the time traveller knows Open Solaris up to 2010. Someone used Linux facilities that are somewhat similar to Solaris Zones and ZFS snapshots (you'll know those, since Solaris 10 from 2005 shipped with that and wasn't even the first) to build an ecosystem for lightweight virtual systems with standardized interfaces. That system is called Docker (and runs on Linux, obviously) OpenSolaris is all but dead, but a community succeeded it, called IllumOS. People from that community decided to attempt to use Solaris Zones, with Linux branded zones (that capability was dropped in the meantime so they had to invest some effort to bring it up again) to reimplement Docker on an IllumOS (ie. OpenSolaris) foundation.

jimpick 12y ago

Nice!

thinkingkong 12y ago

Tldr. Triton lets you run your docker containers in the cloud without worrying about virtual machines or setting up your own paas. Signup here: https://www.joyent.com/lp/preview https://www.joyent.com/lp/preview

hueving 12y ago
So how is this different from the container service from google?
- thinkingkong 12y ago
  Frankly, the technical implementation.
- brianshaler 12y ago
  One interesting bit from the article: "Does this mean that I'll be able to DTrace my Linux apps in a Docker container?" (yes) Technically you can run dtrace on various platforms, and I don't know enough to tell you specific reasons why you would rather use dtrace on smartos than linux. I got the impression a few years ago the ports weren't as good. (perhaps relating to the linux port not yet being a kernel module?)
  - lloydde 12y ago
    Debugging LX branded zones on SmartOS https://www.youtube.com/watch?v=6oIBiWdh41c https://www.youtube.com/watch?v=6oIBiWdh41c "A gritty how-to on debugging LX branded zones on SmartOS presented internally at Joyent on 1/30/15. While given to an internal audience, there is quite a bit of information here that may be interesting to the broader community"
mcescalante 12y ago
Thanks, I spent a minute skimming through this post and ended up coming here hoping someone wrote a tl;dr like this. Must be challenging to write effective copy for something like this (not being sarcastic)
12y ago

phildougherty 12y ago

"You can run your Docker containers across entire data centers without ever creating a "cluster" as other IaaS providers would have you do." This is a little unclear. I get that you don't have to create a "cluster" because you're going to use the API to launch containers on existing dedicated hardware, but how is that a WIN for your finance team? I guess I'm missing something.. but wouldn't you have to already have systems online that you're paying for that these containers can be launched on? From: https://www.joyent.com/blog/docker-bake-off-aws-vs-joyent https://www.joyent.com/blog/docker-bake-off-aws-vs-joyent

thinkingkong 12y ago
The difference for the operator is: Baremetal -> smartos -> container vs Baremetal -> hypervisor -> virtual machine -> container. Theres root safety in the smartos implementation of docker. So they can do multi-tenancy. For the customer, you dont have to provision entire virtual machines to run docker containers.
- phildougherty 12y ago
  Is it multi-tenancy of various Joyent customers on the same hardware? So you get containers placed somewhere in their data center on shared hardware with other customers? Or do you already need to have dedicated hardware provisioned at Joyent to launch containers on? I'm still unclear on that point, which makes me question how it's saving the end user money.
  - xj9 12y ago
    Yes, you are running on shared hardware, securely, but with no hypervisor. The biggest money-saver is in performance. IO-heavy applications can see anywhere from 5-10x performance improvements[1] by switching from hardware- to OS-virtualization. [1]: http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-zones-kvm-xen/ http://dtrace.org/blogs/brendan/2013/01/11/virtualization-pe...
  - misterbisson 12y ago
    Yes, the the containers are running on bare-metal multi tenant hardware. Some of the savings comes from the performance gains as noted elsewhere, but additional savings comes from not having to pay for and manage an additional layer of virtual machines. There's no need to provision anything other than the containers.

redwood631 12y ago

This is running Linux binaries on Oracle Solaris (which they call SmartOS), and not the real thing that Oracle is still developing, but a fork from 2010 that is maintained by 1/10th of the developers it once had. https://www.openhub.net/p/illumos https://www.openhub.net/p/illumos. This would have been pretty rad 10 years ago, when the world still cared about Solaris. On a minor note: the post doesn't credit Oracle or Solaris, from where more than 90% of their SmartOS code comes from, until Oracle closed their code in 2010.

thinkingkong 12y ago
Thats because Oracle deserves no credit. The illumos project was created as a derivative of opensolaris which was created by Sun. The lineage is actually sort of mind boggling. If we want to be pedantic, we could complain about nobody is giving credit to BSD.
- jimpick 12y ago
  And Bell Labs.
pyre 12y ago
> until Oracle closed their code in 2010 At that point, why credit Oracle instead of Sun? Oracle finalized their deal to buy Sun in January 2010. How much did Oracle contribute between then and the announcement that they were closing their code? Edit: Oracle announced closing OpenSolaris on August 13, 2010. The last "release" of OpenSolaris was June 2009.
- bahamat 12y ago
  Oracle never publicly announced it. The last commit that I see copyrighted by Oracle is https://github.com/illumos/illumos-gate/commit/ea10ff14a02f7788b8ec27b4b21b925542a8cca6 https://github.com/illumos/illumos-gate/commit/ea10ff14a02f7... on August 18, 2010. After that, updates from Oracle silently stop. But your point stands well. Oracle only had a paltry 7-8 months of contributions. In my opinion, and I really don't know who would share this view, since Sun opened Solaris, illumos is now what Solaris was intended to be; it's not the illumos community that forked Solaris, it was Oracle.

josh2600 12y ago

Joyent has solved some really hard problems to make this work. I was lucky enough to get the chance to sit down with Bryan last week and talk with him about what this took. It's essentially reimplementing Linux on top of Illumnos, which is no tiny feat. There are still messy hurdles to running Docker in production, but it's clear that Joyent has really tried to make something awesome here. As someone who works deeply in the future-container space, I applaud folks who are taking us deeper down the rabbit hole. I think it's clear that this whole 'back-to-the-future' isolation technology is really cool stuff. Jails/zones/containers have been around forever and I think it's great that we're finally taking advantage of this technology. Edit: at Terminal.com, we want people to push Linux forward, and this is a great example of taking Linux to new and intriguing heights. I did not think we would have Linux on Illumnos in quite this way in 2015 and it's delightful to see. We are all standing on the shoulders of giants and it's great to reach new vistas.

kordless 12y ago
+1 on terminal.com. Love it.
justincormack 12y ago
Isn't it an example of taking Solaris to new and intriguing heights? There is no actual Linux code in the implementation.
- josh2600 12y ago
  That's a great point, but I think the point of what Bryan has been doing is to make Linux work with Zones (and dtrace). That's a primitive that Joyent has wanted to upstream into the Linux kernel for a long time and has never been able to get the necessary consensus around it (similar to OpenVZ's troubles getting their work upstreamed). In short, this is sort of a hack to give you zones on Linux without needing to get zones into the upstream. Yes, there's no linux code, but there is a lot of required understanding of Linux code to make something like this work. It's kinda amazing that they got 64-bit linux to run on top of Illumnos, right? I did not see that coming and maybe that's because I'm ignorant in some capacity, but it's been a pleasant surprise.

mrinterweb 12y ago

Is there any pricing information for Triton yet?

jholmes83 12y ago
Pricing here: https://www.joyent.com/blog/expanded-container-service-preview https://www.joyent.com/blog/expanded-container-service-previ...
- mrinterweb 12y ago
  Thanks for finding that link. That is a very attractive pricing model. I am currently running about 10 VPS running about 25 containers. The costs of the VPS hosting is affordable, but managing those VPS hosts, deployment, balancing, discovery, and scaling are the hard parts. I'm looking forward to learning more about Triton.

themgt 12y ago

Didn't see the actual repo linked: https://github.com/joyent/sdc-docker https://github.com/joyent/sdc-docker "A Docker Engine for SmartDataCenter, where the whole DC is exposed as a single docker host. The Docker remote API is served from a 'docker' core SDC zone (built from this repo). ... Disclaimer: This is still very much alpha. Use at your own risk!"

trentmick 12y ago
Thanks for the link. I've updated the "current state" notes to better reflect reality. https://github.com/joyent/sdc-docker#current-state https://github.com/joyent/sdc-docker#current-state
ochoseis 12y ago
To me it seems like Triton is about shoehorning Linux into Solaris/SmartOS containers. While this seems great for accelerating adoption, why not just use SmartOS-specific Dockerfiles/repos in conjunction with sdc-docker (Not directly addressing the parent, more of a general question)? Imagine all of the crazy bugs and edge cases that will arise when trying to run Linux binaries on UNIX. I'm surely making some assumptions, but the primary languages/developers that something like this targets are probably like Java, Javascript/Node.js, Ruby, Python, C...which are probably all portable enough to run on SmartOS without much/any modification.
- wmf 12y ago
  Docker multiarch support isn't finished yet, and getting all the world's Dockerfiles rebuilt on SmartOS is a big task. There are also different philosophies about whether container images should be distributed as source (Dockerfiles) or as binaries. It looks like Joyent decided to run Linux/x86-64 binaries to be on the safe side.

crashoverdrive 12y ago

I feel like perhaps I'm the only one concerned about this: (Somehow, “SmartOS + LX-branded zones + SmartDataCenter + sdc-portolan + sdc-docker” was a bit of a mouthful.) Not only is it a mouthful, it means there is a much wider space for things to go wrong.

tracker1 12y ago
Compared to say "hypervisor + Linux + kubernetes + weave + etcd + lxc-docker" ...? Not to mention all the pieces that work together for lxc-docker.

23david 12y ago

Excited to see this. Congrats to the team on the release!

bau5 12y ago

I saw the name and my desire to read it plummeted. The same person wrote http://www.joyent.com/blog/the-power-of-a-pronoun http://www.joyent.com/blog/the-power-of-a-pronoun which led to one of the most talented developers of node core and libuv leaving the project for a while.

rubiquity 12y ago
What relevance does that have to this? It's sad that your post is even being upvoted.
stuaxo 12y ago
Wow, bloody hell - hadn't read that before.
- cookiecaper 12y ago
  Me either. Wondering how I missed all that drama but not particularly sad I did.
codygman 12y ago
The power of a pronoun is a great article, if anything him writing it should be a plus. The only way it is a negative is if you feel gendered pronouns don't further sexism, but I don't see how you could support that argument.
- crazy1van 12y ago
  Does Joyent refuse to hire speakers of Spanish, French, Italian, Hindi, or any of the other dozens of languages that have gender throughout the entire grammar?
  - nosequel 12y ago
    You are missing the entire point. It is one thing to not allow gender pronouns (what you are suggesting) versus punishing someone who specifically set out to change gender-neutral pronouns into male dominated ones (the Joyent case). REAL BIG difference.

DannoHung 12y ago

Uh, so I've never really heard of SmartOS or Joyent's SDC before now... This stuff looks pretty fucking unreal. What's the catch? It looks like it solves so many problems with clustered container deployment. Exposing a whole datacenter as a single Docker host seems like the end-game to me. Am I missing something? I'm a little low on sleep today. edit: To answer my own question a little: I guess the docker-sdc isn't quite fully baked yet. That's not really a huge issue since it's still a preview, I think?

bananaoomarang 12y ago
Nope, SmartOS is pretty neat.
- ssmoot 12y ago
  Ditto. We were early adopters at work. KVM + ZFS and practically no setup is ace.
jgrowl 12y ago
I'm right there with you. It feels like the beginning of skynet
pannon 12y ago
No catch, it is a truly innovative/disruptive technology.
_delirium 12y ago
The main catch (which may also be a plus) is that the Joyent set of technologies is quite opinionated about how to manage a cloud, so you more or less have to "buy in" and do things its way. Your node OS will be SmartOS, nodes will be PXE-booted, your filesystem will be ZFS, the network topology will work in a specific way, etc. More practically, it also takes some time to be proficient in operating it (or even just get it working), because there are quite a few moving parts. Getting a SmartDataCenter deployment running, and then things like Manta and now sdc-docker running on top of it, is more than an afternoon's work. But if you're willing to buy in to the system and its choices fit your needs, it's really well engineered, imo much more "done right" than e.g. OpenStack is.

kstenerud 12y ago

Is there any further work going on to support AMD in the KVM driver? I remember the codebase having sat idle for a few years last time I checked...

snw 12y ago
While sadly not merged upstream yet, it works very well. I've been running custom SmartOS "eait" builds from http://imgapi.uqcloud.net/builds http://imgapi.uqcloud.net/builds (which include AMD support) for some time now without any issues. The source can be found at https://github.com/arekinath/smartos-live https://github.com/arekinath/smartos-live

themgt 12y ago

I find these to be pretty weasel words: Is Triton open source? Yes. Triton Elastic Container Infrastructure is the commercial offering built on the top of the open-source SmartDataCenter cloud infrastructure management platform. In addition to the open source components, Triton includes support, a DevOps portal, and intellectual property indemnification only available to commercial customers. In more plain English, "Yes. Triton is open source just like Mac OS X" https://www.joyent.com/developers/triton-faq#os https://www.joyent.com/developers/triton-faq#os

lloydde 12y ago
That's a good thing to clarify. It's not an open core system like Mac OS X. SmartDataCenter is open source and Triton is our product branding. https://github.com/joyent/sdc https://github.com/joyent/sdc There is work in progress to clean up and open source the self-serve portal and its dependencies. The propriety bits are not ours: "The Joyent SDC USB images offered to our customers contain certain firmware binaries for use with Joyent-branded hardware. We cannot make that software generally available in the open source builds; however, it is available to all who have purchased Joyent-branded hardware and is of no use without that hardware." https://docs.joyent.com/sdc7/obtaining-software https://docs.joyent.com/sdc7/obtaining-software
- DannoHung 12y ago
  What does the bit about security patches and minor and major updates mean? The Product version includes automatic updates and the open source version you manage yourself? Or something else?
  - lloydde 12y ago
    Would you quote the bits here, so I can clarify with the product side of the house? We have not and never will let anything jeopardize the continued security of any SDC deployments. SDC having update channels is under active development: 'As of sdcadm version 1.5.5 and the release-20150319, SmartDataCenter has preliminary support for update "channels". A "channel" is separate stream of built SDC components with different stability characteristics.' https://github.com/joyent/sdc/blob/master/docs/operator-guide/update.md https://github.com/joyent/sdc/blob/master/docs/operator-guid... I image that the support channel will be the most conservative.

yonran 12y ago

Very cool. Is the next step integration with Manta so you can specify an environment of Linux executables to map over your data?

dap 12y ago
Yes, that's definitely the direction that we're headed. Manta allows users to spin up large numbers of transient containers around their data (i.e., without moving data around) in order to do map-reduce operations, and without having to think about the container management. Today, these compute tasks are specified as scripts that run inside a well-stocked SmartOS container, plus some optional assets to download (for custom binaries or other data). The obvious next step would be to let people specify they want to run an LX zone (rather than a SmartOS zone) so that they can run GNU/Linux binaries. Then we can consider whether it makes sense to phrase the task as a Docker image. That might well be a more natural way to incorporate a larger bunch of binaries and other shared data. Architecturally, all of this should be relatively straightforward (famous last words!).

batbomb 12y ago

> When Docker first rocketed I thought this was an interesting way to start off the post.

Bedon292 12y ago

This looks like a very nice way to deploy containers. I see the per hour prices (https://www.joyent.com/blog/expanded-container-service-preview https://www.joyent.com/blog/expanded-container-service-previ...) but is there any pricing on bandwidth? I imagine there will be eventually, right?

adamtaro 12y ago
I'd suggest looking at this page as an indicator (https://www.joyent.com/public-cloud/pricing https://www.joyent.com/public-cloud/pricing -- Bandwidth tab): all inbound traffic free, outbound is 12 cents/GB after the first GB, cheaper at higher tiers.