21 ms·

Show HN: Magic Sandbox – learn Kubernetes on real infrastructure

392 points by mstipetic 8y ago

tootie 8y ago
This is a really cool platform, but it kinda exposes the extremely high barrier of entry to getting running with k8s. It's a very powerful and useful platform, but it doesn't seem to strike the right balance with it's abstractions to actually make intuitive sense to developers.
- striking 8y ago
  That's because, for the vast majority of developers, there's no point in using k8s. It's really only necessary when you have a big app (or a few instances of a few big apps) that will have lots of containers and lots of management required. Please. Don't use k8s as a golden hammer. Docker alone (or with docker-compose) will be just fine for most people, especially when paired with a basic CI pipeline like Drone or Jenkins. (If you're reading this, thinking "but it makes installing and managing services I depend on easier!" it doesn't obviate you of that responsibility entirely, not in production. You still have to make sure those services deploy properly, you still have to manage them to some degree... for some time, it was considered inadvisable to run your DB in-cluster because of how many ways it could fail.)
  - manigandham 8y ago
    Kubernetes still provides easy deployments, monitoring, logging, load-balancing, scaling/high-availability that is easy to take advantage of and can work with even a single app.
    - gm-conspiracy 8y ago
      But, why do you need k8s to do those things, particularly for a single app? It seems like a very complex architecture to ensure reliability.
      - manigandham 8y ago
        You don't, but K8S has it built-in which removes overhead. If you're installing K8S yourself then it's probably not worth it, but if you're using managed services with free master nodes, why not take advantage of it? You can use it with a single worker node without problems.
  - deforciant 8y ago
    I am a solo developer and I have been running my side projects on Kubernetes (GKE) for several years now. It helped me a lot to focus on things that I like: development instead of ops. I never had to SSH into any machine or do things manually. I just tag a github release and in few minutes I get my application updated. With Docker I believe it would be lot more complicated, no zero-downtime upgrades, no load balancing, OS vulnerabilities. Then, when I would need to add a new service into my setup, I would have to yet again go into the host or multiple hosts :) So no, I don't believe people who say that for small projects you shouldn't use K8s. It saves a lot of time and helps you not to burn out on your side projects as it takes the pain of running a service away. P.S. I am an author of Keel https://github.com/keel-hq/keel https://github.com/keel-hq/keel, it's a relatively successful CD solution, I hope it saved a lot of hours for other companies too :)
    - XVincentX 8y ago
      Question: Why not Heroku or any other hosted service like that? They seem to give you the right compromise for solo developers as well as mid size applications. If it was a learning experience than I totally get your choice — otherwise I'd be interested in the motivations! Thanks!
      - deleted 8y ago
        [deleted]
      - deforciant 8y ago
        I wanted my own ports with non http services as I had a grpc endpoint :) not sure how it is now but heroku only offered http.
      - meddlepal 8y ago
        Personally, Heroku has too much magic around the build packs and what not. Kubernetes for stateless apps is easy: 1. Create a Docker image with your app. 2. Write about 20 lines of YAML for a Deployment 3. Write maybe 10 lines of YAML for a Service Done. You can automate the entire roll out via Travis and using some `sed` replacements on the Deployment. I have got blue-green roll outs working with two deployments and two service descriptors and maybe 30 lines of `bash`.
  - solatic 8y ago
    For developers, or for operators? Orchestration makes it trivial to recover from the downtime of individual nodes, unintentional (crashes) or intentional (managing updates of the node's underlying OS). Making load balancing, (basic) service discovery, storage as part of the deployment artifact drastically reduces deployment risk. We can have a discussion about which orchestration platform is best, and we can have a discussion on whether you need orchestration if you're a two-person startup using mostly a lot of SaaS and you have Really Only One Thing running in your production environment. But doubting the value of orchestration? Really?
    - jwr 8y ago
      Nothing is "trivial" about orchestration. It introduces entire new layers of complexity, new failure modes, as well as second-order effects that are poorly understood. People are not afraid enough of complexity. I also think that solutions like Kubernetes should be used only when you really have to. And even then very carefully and never without a tool like Chaos Monkey.
      - specialp 8y ago
        Orchestration does introduce complexity but it does so with a large base of users, developers and general ecosystem. As developers and systems implementors we are already sitting on top of a huge stack of what could be an impossibly complex system (CPUs, operating systems, networks) But people have worked on their layer and made it robust and everyone benefits. With that said we have been running K8s for 2 years now in production and have not once had an issue that we could attribute to it. So yes we do have another abstraction (complexity) layer, but to us the complexity does not equate to complicated problems. The benefits gained from the seamless redundancy, discovery, scalability, encapsulation and resource management have not cost us anything. If we had a shaky hard to understand stack that failed often I'd 100% agree.
        raesene9 8y ago
        One other complexity that k8s (or similar) introduces is more security concerns. If you've been runnning for 2 years now, I'm sure you'll have seen the changes required to secure Kubernetes from the "old days". Managing the multiple TLS CAs, RBAC, PSP, Network Policy etc is non-trivial.
  - tootie 8y ago
    My company is currently struggling a bit with building out a k8s platform. The benefit we seek is to have elastic compute capacity a la any cloud provider and have it be on-prem, strictly governed but also fully self-service. I might have an app with only 100-1000 users total, but I'd still like to get georedundancy and autoscaling to minimize support costs. That's what container orchestration is supposed to provide but it's still a windy path.
- manigandham 8y ago
  The barrier is only as high as the functionality you want to take advantage of. Running a single container as an app is 1 line with kubectl. Scaling it up in replicas is another line. Opening it up to a load balancer is another line. In 3 lines you have a fully distributed load balanced public app with monitoring and logging built in and easy deployments to the next version. You can then graduate slowly to using YAML definitions for the parts you care about, and then put them in source control for repeatable deployments. It's a very nice system and arguably even better for simple isolated deployments than more complex apps. Kubernetes really is no more complicated than any other major thing in the industry. Trying explaining a new programming language or framework to someone... is that easy?
  - gm-conspiracy 8y ago
    You are assuming k8s is already setup and running, right? Say I want to deploy my application via k8s to bare-metal, Google, and Amazon (so I am not dependent on one provider)...what does that look like? It seems like we are moving away from standards towards proprietary abstractions with regard to cloud environments.
    - manigandham 8y ago
      Yes, if you have to install it from scratch then yes, it's more work. You can install from the source binaries or use KOPS or Rancher or a dozen other installers to set it up in a few clicks or CLI commands. If you're on the clouds, then stick with the managed offerings since they actually give you free master nodes. Federation/multi-cluster is being reworked so multiple K8S clusters around the world on different providers can act as one, or you can use Rancher or some bash scripts to accomplish most of it today. Also K8S is actually moving towards standards as every release includes a new interface for things that were previously proprietary, and now just use plugins for whatever provider you want.
    - xur17 8y ago
      > You are assuming k8s is already setup and running, right? I've only used Google's managed k8s solution, but setting up the actual cluster is another 30 seconds of clicking 2 buttons on a website. > Say I want to deploy my application via k8s to bare-metal, Google, and Amazon (so I am not dependent on one provider)...what does that look like? As with any other multi-cloud solution, you need to decide how you're going to handle your dbs and other persistent data stores, but it's as simple as spinning up a cluster (rather than a vm) on each service, and then deploying to each. With a little bit of upfront work, you get a ton of extra features. I've used Elastic Beanstalk on AWS, and running an app on Kubernetes is easier, even including the cluster setup time, and as a bonus, your deployment configurations aren't tied to a single vendor (you can pickup your k8s configs and more to another provider if needed).
- djsumdog 8y ago
  This is the problem I have with almost all container orchestration systems (DC/OS, K8s, etc.) None of them can go from 1 node running all your apps to 100, in an easy straight forward manner. I wrote about this not too long ago: https://penguindreams.org/blog/my-love-hate-relationship-with-docker-and-container-orchestration-systems/#zero-to-a-hundred https://penguindreams.org/blog/my-love-hate-relationship-wit... I've also worked at both Kubernetes shops and DCOS/marathon shops and I honestly don't understand why people are choosing K8s. I'm sure a lot of it is the marketing and the power of Google behind it, but our DC/OS cluster had hundreds of nodes and could scale applications really well. Plus I found the marathon json files way less confusing. (Plus if you really want, you can use k8s to schedule containers on DC/OS, alongside marathon). That being said, that DCOS platform team was 11 people and they had tons of custom scripts, wiki pages, labels and specific networking ingress/egress points that all worked together. (You could even use a label to specify if you wanted to run containers on the local VMWare cluster or in AWS; with nodes that auto-scaled out into AWS in high load). I've met people at smaller startups who've gone the Nomad route instead, which seems a lot more sane in many ways, but still requires pretty careful planning and setup for large deploys.
  - asdkhadsj 8y ago
    So I'm heading a shop with some serious legacy deployment woes. I've got plans for everything but scaling. Now, we're likely to hire out Ops eventually, but for now I'd like to learn devops (backend eng currently) and cut my teeth by getting the ball rolling on a slightly more robust setup. With that said, what do you recommend? We're a small shop, and our auto scaling needs a very very minimal. We're non-public facing, so our user base is pretty predictable. Yet, I'd like to migrate us to a more modern and mature system. Naturally I was looking towards K8s because of the overwhelming support I see for K8s on HN/etc, but your comment makes me question that. Honestly, it's the vast options that really screw with my head. There are just so many tools. And where they lack is not immediately evident from the outside. It's difficult to pick a direction. At the moment I'm reviewing Nomad, as it sounds interesting to work with our existing by-hand infrastructure.
    - politelemon 8y ago
- wenc 8y ago
  K8S has many different abstractions and it's hard to get a feel for all of them from just reading the docs. I found the PluralSight videos (subscription required) [1] helpful for getting a comprehensive introduction to K8S, along with coverage of the gotchas. The videos also helped me discern the kinds of problems K8S solves and the kinds it doesn't. If you need to deal with scale issues, the complexity of bespoke K8S may be unavoidable complexity. Otherwise, it is overkill. (exception: managed K8S services like GKE, AKS provide middle grounds) [1] https://app.pluralsight.com/library/courses/getting-started-kubernetes/table-of-contents https://app.pluralsight.com/library/courses/getting-started-...
- specialp 8y ago
  I think the only high barrier of entry is actually getting a cluster going outside of Minikube. The abstractions for pods, services, replica-sets etc are pretty easy to understand and the alternative for scaling a service/making it discoverable/highly available has a higher barrier of entry IMO. We just don't see it that way because we have lived it. If I told someone that had no experience with dev-ops how to do this in K8s vs setting up load balancers/DNS/health checking it would be much more difficult. The barrier to entry that is currently high (setting up a cluster) is getting much easier with projects and services on many cloud platforms doing that for you.
  - tootie 8y ago
    Pods, services and replica sets are not complicated in and of themselves. The problem is that they don't reflect the mental model of a user at all. I just care about instances and load balancers.
    - tybit 8y ago
      OTOH I don’t care about instances and load balancers at all(ideally), I just care about services.
- djhworld 8y ago
  I have a few raspberry pis at home that I run a few Docker containers on, mostly personal stuff/tools/scripts/servers that I update infrequently but use a lot. Any software I write goes into a container, I like that abstraction. I really want to reduce the friction between developing/maintaing something and deploying/running it. Right now I just have a script for each service that stops the docker container, pulls the latest version from a private repo and starts it again. But that "solution" is getting cumbersome to manage as I add more containers to run and I have to remember what Raspberry Pi each container is running on if I'm running a service that I don't need to have on 2 nodes. I tried to learn Kubernetes a few weeks ago by installing it on one of my Pis, but I quickly realised the learning curve was way beyond what I wanted to be spending my time with (right now - I realise this is a feeble excuse) - especially for home stuff sitting behind a private network. I've been looking at Docker Swarm recently instead which looks a lot simpler. Maybe it will be a good stepping stone to grokking Kubernetes.
  - Drdrdrq 8y ago
    A side note, since you mention RPi-s... Do you have any trouble with SD cards (or USB sticks) breaking down on power loss? It seems like a recurring theme and I'm curious how you deal with it in your setup? I am thinking of getting a NAS for my projects instead.
- Benjamin_Dobell 8y ago
  I must say, I'm presently migrating a client from Dokku to Kubernetes and finding K8s tooling to be severely lacking. I've plenty of experience with Docker and Docker Compose; both for development, and also running on cloud infrastructure (although admittedly not Docker Swarm). I also have some limited experience with Amazon CloudFormation and ECS. I'm no stranger to performing Docker migrations, having previously been responsible for heading up migrations to Docker cloud deployments from "old school" customised on-site rack deployments, VPS LAMPP deployments, and "install everything on this one machine, what's reproducibility?" setups. However, I'm finding that migrating to Kubernetes is by far the worst experience I've had when compared to the alternatives that I've tried. I know cluster deployment/management tends to be a common complaint, however I actually found that to be not too difficult. Thanks in large part to kops. Instead my biggest gripes are: 1. Supposed Immutability Kubernetes treats certain "Dockerisms" (e.g. image tags) as immutable, when they're clearly not. This leads to many subtle "gotchas" and associated work-arounds that are non-intuitive. 2. Templating/substitution (or lack there-of) The argument from the K8s core team seems to be that templating result in non-reproducible deployments. However, I find the opposite to be true. I don't want to maintain slightly different resources for all my different environments and have to manually keep them in sync. That's just asking for trouble. I want a single source of truth. I know Helm (Charts) can be used for templating. However, I find Helm to be over-kill, particularly the Tiller runtime requirement. Helm v3 sounds very promising though, and I have every intention of using Helm more extensively once released. 3. Tooling immaturity / Lack of standardisation I seem to be writing a lot of (simple) bash scripts to solve trivial problems, no doubt others are doing similar things and I'm reinventing the wheel. With CloudFormation/ECS or Docker Compose deployments I don't seem require any scripts at all. Problems can be solved trivially in a declarative way, or by running a single command - of course this approach is supplemented by project-specific documentation. I suppose this largely comes down to K8s flexibility; there's so many different ways to achieve something. I'm not necessarily suggesting stripping away that flexibility, as it's part of what makes K8s great. However, there's a lot that could be simplified for 90% of use-cases, particularly if there were more standardisation and widely agreed upon "best practices". I hope this will come in time. 4. Debugging This isn't at all a fault of K8s itself, just platform immaturity. However, despite the fact K8s seems to be emerging as the winner of the container orchestration war, most alternatives seem to offer significantly better tooling e.g. Docker Compose has proper integration into Jetbrains IDEs, including "remote" debugging of apps running in containers. Jetbrains K8s plugin is currently limited to interpretation of YAML/JSON resource semantics. Don't get me wrong I'm using K8s because I think it's hugely powerful/reliable, and I'm betting on it long-term. Presently, it's truly fantastic from an ops perspective, just significantly less so from a development perspective.
- numbsafari 8y ago
  I mean... keep in mind that this is a vendor who will make money by convincing you that it's complicated and requires a lot of extra training. So, you gotta take it with a grain of salt. Personally, I've found it to be pretty easy to grasp so far and I'm not the smartest person in the world.
Operyl 8y ago
Super cool! Is only 1 level available right now? EDIT: Further, the front page marketing and actual app don't line up 1:1 very well when it comes to listing lessons.
- mstipetic 8y ago
  Yeah, sorry, we're still building it up, we didn't plan for this big of a reception, just wanted to test if there's any interest out there
  - dfcowell 8y ago
    Good problem to have. :)
mstipetic 8y ago
Hey guys, we're really trying to keep up, but we've gone to 300+ servers in a few minutes. I'm happy to answer any questions here you may have
- earth2mars 8y ago
  what does that mean? are you going to support more servers or you can't provide more servers?! Thank you for this. looks interesting. would love to get hands-on but it says "We are booting up your cluster!"
  - mstipetic 8y ago
    Hey we hit our hard limit of 500 servers, I'm trying to shut down old ones, really sorry about that, but traffic is just crazy
    - nmg 8y ago
      Sounds like the good kind of problem! Let me chime in and say thank you for implementing and sharing this.
      - mstipetic 8y ago
        Thank you very much! Rationally I know this is a good thing, but doesn't feel that way right now
- deleted 8y ago
  [deleted]
- Serow225 8y ago
  Hug of death! Congrats :)
mwj 8y ago
How is this different to www.katacoda.com?
- vishnu_ks 8y ago
  I would also like to know. I initially thought it was katacode.
  - mstipetic 8y ago
    we really love katacoda! we're a bit more oriented on trying to visually explain what's happening (not there yet) and we're trying to give you a very high level of interactivity
Arubis 8y ago
Super interested to see how this works once the hug has subsided. This is potentially fortuitous timing; my employer wants me to cover most of this material by end of quarter.
- mstipetic 8y ago
  Thanks a lot, really didn't expect this kind of traffic. If you ever want to get in touch you can reach me at mislav@<website-domain> or through our slack that's on our landing page and inside console
theyoungwolf 8y ago
Is this for someone who has no experience with any of this or is there prior knowledge required?
- mstipetic 8y ago
  Hey, we currently host only an intro course but we're building out advanced courses now. So no kubernetes prior knowledge is necessary, but we assume you're a developer with some experience
Drdrdrq 8y ago
> 1 year early bird access > Get 1 year unlimited access to MSB Premium including all lessons and weekly releases. > This is a non-recurring, one time payment. You probably need to fix this, or at least clarify? Is it a yearly sum or one-time payment?
- mstipetic 8y ago
  Hm, we're doing a presale with unlimited access for 1 year in one payment
  - Drdrdrq 8y ago
    That sounds good, but I would suggest clarifying this. Good luck btw!
tjungblut 8y ago
Will it be possible to contribute lessons to your platform? I find conferences a terrible place to share knowledge on K8s, this makes it a lot easier and hands-on.
- mstipetic 8y ago
  Yes! I'm working on an editor right now, feel free to contact me at mislav at magicsandbox dot com if you're interested
Rafuino 8y ago
Can't find it on your site... where are you spinning up all these servers for the sandbox? GCP? Azure? AWS? Somewhere else?
- mstipetic 8y ago
  Digital Ocean currently. Just running minikube and kubectl proxy for the feed
StavrosK 8y ago
I found Kubernetes to be easier to learn than the docs would suggest, and I wrote up a short article as a high-level overview. I hope it helps someone, and that it saves you a lot of time: https://www.stavros.io/posts/kubernetes-101/ https://www.stavros.io/posts/kubernetes-101/ You may want to spend two minutes reading this before going into the post's website, as an overview.
lgregg 8y ago
This is awesome, I been looking for something like this.
reactdom 8y ago
If any one is look for Kubernetes books or Kubernetes tutorials: https://reactdom.com/kubernetes https://reactdom.com/kubernetes
alpb 8y ago
What do the Facebook Google Microsoft logos at the bottom of this page mean? Are these companies Magic Sandbox customers?

392 points by mstipetic 8y ago

This is a really cool platform, but it kinda exposes the extremely high barrier of entry to getting running with k8s. It's a very powerful and useful platform, but it doesn't seem to strike the right balance with it's abstractions to actually make intuitive sense to developers.

striking 8y ago
That's because, for the vast majority of developers, there's no point in using k8s. It's really only necessary when you have a big app (or a few instances of a few big apps) that will have lots of containers and lots of management required. Please. Don't use k8s as a golden hammer. Docker alone (or with docker-compose) will be just fine for most people, especially when paired with a basic CI pipeline like Drone or Jenkins. (If you're reading this, thinking "but it makes installing and managing services I depend on easier!" it doesn't obviate you of that responsibility entirely, not in production. You still have to make sure those services deploy properly, you still have to manage them to some degree... for some time, it was considered inadvisable to run your DB in-cluster because of how many ways it could fail.)
- manigandham 8y ago
  Kubernetes still provides easy deployments, monitoring, logging, load-balancing, scaling/high-availability that is easy to take advantage of and can work with even a single app.
  - gm-conspiracy 8y ago
    But, why do you need k8s to do those things, particularly for a single app? It seems like a very complex architecture to ensure reliability.
    - manigandham 8y ago
      You don't, but K8S has it built-in which removes overhead. If you're installing K8S yourself then it's probably not worth it, but if you're using managed services with free master nodes, why not take advantage of it? You can use it with a single worker node without problems.
- deforciant 8y ago
  I am a solo developer and I have been running my side projects on Kubernetes (GKE) for several years now. It helped me a lot to focus on things that I like: development instead of ops. I never had to SSH into any machine or do things manually. I just tag a github release and in few minutes I get my application updated. With Docker I believe it would be lot more complicated, no zero-downtime upgrades, no load balancing, OS vulnerabilities. Then, when I would need to add a new service into my setup, I would have to yet again go into the host or multiple hosts :) So no, I don't believe people who say that for small projects you shouldn't use K8s. It saves a lot of time and helps you not to burn out on your side projects as it takes the pain of running a service away. P.S. I am an author of Keel https://github.com/keel-hq/keel https://github.com/keel-hq/keel, it's a relatively successful CD solution, I hope it saved a lot of hours for other companies too :)
  - XVincentX 8y ago
    Question: Why not Heroku or any other hosted service like that? They seem to give you the right compromise for solo developers as well as mid size applications. If it was a learning experience than I totally get your choice — otherwise I'd be interested in the motivations! Thanks!
    - deleted 8y ago
      [deleted]
    - deforciant 8y ago
      I wanted my own ports with non http services as I had a grpc endpoint :) not sure how it is now but heroku only offered http.
    - meddlepal 8y ago
      Personally, Heroku has too much magic around the build packs and what not. Kubernetes for stateless apps is easy: 1. Create a Docker image with your app. 2. Write about 20 lines of YAML for a Deployment 3. Write maybe 10 lines of YAML for a Service Done. You can automate the entire roll out via Travis and using some `sed` replacements on the Deployment. I have got blue-green roll outs working with two deployments and two service descriptors and maybe 30 lines of `bash`.
- solatic 8y ago
  For developers, or for operators? Orchestration makes it trivial to recover from the downtime of individual nodes, unintentional (crashes) or intentional (managing updates of the node's underlying OS). Making load balancing, (basic) service discovery, storage as part of the deployment artifact drastically reduces deployment risk. We can have a discussion about which orchestration platform is best, and we can have a discussion on whether you need orchestration if you're a two-person startup using mostly a lot of SaaS and you have Really Only One Thing running in your production environment. But doubting the value of orchestration? Really?
  - jwr 8y ago
    Nothing is "trivial" about orchestration. It introduces entire new layers of complexity, new failure modes, as well as second-order effects that are poorly understood. People are not afraid enough of complexity. I also think that solutions like Kubernetes should be used only when you really have to. And even then very carefully and never without a tool like Chaos Monkey.
    - specialp 8y ago
      Orchestration does introduce complexity but it does so with a large base of users, developers and general ecosystem. As developers and systems implementors we are already sitting on top of a huge stack of what could be an impossibly complex system (CPUs, operating systems, networks) But people have worked on their layer and made it robust and everyone benefits. With that said we have been running K8s for 2 years now in production and have not once had an issue that we could attribute to it. So yes we do have another abstraction (complexity) layer, but to us the complexity does not equate to complicated problems. The benefits gained from the seamless redundancy, discovery, scalability, encapsulation and resource management have not cost us anything. If we had a shaky hard to understand stack that failed often I'd 100% agree.
      - raesene9 8y ago
        One other complexity that k8s (or similar) introduces is more security concerns. If you've been runnning for 2 years now, I'm sure you'll have seen the changes required to secure Kubernetes from the "old days". Managing the multiple TLS CAs, RBAC, PSP, Network Policy etc is non-trivial.
- tootie 8y ago
  My company is currently struggling a bit with building out a k8s platform. The benefit we seek is to have elastic compute capacity a la any cloud provider and have it be on-prem, strictly governed but also fully self-service. I might have an app with only 100-1000 users total, but I'd still like to get georedundancy and autoscaling to minimize support costs. That's what container orchestration is supposed to provide but it's still a windy path.
manigandham 8y ago
The barrier is only as high as the functionality you want to take advantage of. Running a single container as an app is 1 line with kubectl. Scaling it up in replicas is another line. Opening it up to a load balancer is another line. In 3 lines you have a fully distributed load balanced public app with monitoring and logging built in and easy deployments to the next version. You can then graduate slowly to using YAML definitions for the parts you care about, and then put them in source control for repeatable deployments. It's a very nice system and arguably even better for simple isolated deployments than more complex apps. Kubernetes really is no more complicated than any other major thing in the industry. Trying explaining a new programming language or framework to someone... is that easy?
- gm-conspiracy 8y ago
  You are assuming k8s is already setup and running, right? Say I want to deploy my application via k8s to bare-metal, Google, and Amazon (so I am not dependent on one provider)...what does that look like? It seems like we are moving away from standards towards proprietary abstractions with regard to cloud environments.
  - manigandham 8y ago
    Yes, if you have to install it from scratch then yes, it's more work. You can install from the source binaries or use KOPS or Rancher or a dozen other installers to set it up in a few clicks or CLI commands. If you're on the clouds, then stick with the managed offerings since they actually give you free master nodes. Federation/multi-cluster is being reworked so multiple K8S clusters around the world on different providers can act as one, or you can use Rancher or some bash scripts to accomplish most of it today. Also K8S is actually moving towards standards as every release includes a new interface for things that were previously proprietary, and now just use plugins for whatever provider you want.
  - xur17 8y ago
    > You are assuming k8s is already setup and running, right? I've only used Google's managed k8s solution, but setting up the actual cluster is another 30 seconds of clicking 2 buttons on a website. > Say I want to deploy my application via k8s to bare-metal, Google, and Amazon (so I am not dependent on one provider)...what does that look like? As with any other multi-cloud solution, you need to decide how you're going to handle your dbs and other persistent data stores, but it's as simple as spinning up a cluster (rather than a vm) on each service, and then deploying to each. With a little bit of upfront work, you get a ton of extra features. I've used Elastic Beanstalk on AWS, and running an app on Kubernetes is easier, even including the cluster setup time, and as a bonus, your deployment configurations aren't tied to a single vendor (you can pickup your k8s configs and more to another provider if needed).
djsumdog 8y ago
This is the problem I have with almost all container orchestration systems (DC/OS, K8s, etc.) None of them can go from 1 node running all your apps to 100, in an easy straight forward manner. I wrote about this not too long ago: https://penguindreams.org/blog/my-love-hate-relationship-with-docker-and-container-orchestration-systems/#zero-to-a-hundred https://penguindreams.org/blog/my-love-hate-relationship-wit... I've also worked at both Kubernetes shops and DCOS/marathon shops and I honestly don't understand why people are choosing K8s. I'm sure a lot of it is the marketing and the power of Google behind it, but our DC/OS cluster had hundreds of nodes and could scale applications really well. Plus I found the marathon json files way less confusing. (Plus if you really want, you can use k8s to schedule containers on DC/OS, alongside marathon). That being said, that DCOS platform team was 11 people and they had tons of custom scripts, wiki pages, labels and specific networking ingress/egress points that all worked together. (You could even use a label to specify if you wanted to run containers on the local VMWare cluster or in AWS; with nodes that auto-scaled out into AWS in high load). I've met people at smaller startups who've gone the Nomad route instead, which seems a lot more sane in many ways, but still requires pretty careful planning and setup for large deploys.
- asdkhadsj 8y ago
  So I'm heading a shop with some serious legacy deployment woes. I've got plans for everything but scaling. Now, we're likely to hire out Ops eventually, but for now I'd like to learn devops (backend eng currently) and cut my teeth by getting the ball rolling on a slightly more robust setup. With that said, what do you recommend? We're a small shop, and our auto scaling needs a very very minimal. We're non-public facing, so our user base is pretty predictable. Yet, I'd like to migrate us to a more modern and mature system. Naturally I was looking towards K8s because of the overwhelming support I see for K8s on HN/etc, but your comment makes me question that. Honestly, it's the vast options that really screw with my head. There are just so many tools. And where they lack is not immediately evident from the outside. It's difficult to pick a direction. At the moment I'm reviewing Nomad, as it sounds interesting to work with our existing by-hand infrastructure.
  - politelemon 8y ago
wenc 8y ago
K8S has many different abstractions and it's hard to get a feel for all of them from just reading the docs. I found the PluralSight videos (subscription required) [1] helpful for getting a comprehensive introduction to K8S, along with coverage of the gotchas. The videos also helped me discern the kinds of problems K8S solves and the kinds it doesn't. If you need to deal with scale issues, the complexity of bespoke K8S may be unavoidable complexity. Otherwise, it is overkill. (exception: managed K8S services like GKE, AKS provide middle grounds) [1] https://app.pluralsight.com/library/courses/getting-started-kubernetes/table-of-contents https://app.pluralsight.com/library/courses/getting-started-...
specialp 8y ago
I think the only high barrier of entry is actually getting a cluster going outside of Minikube. The abstractions for pods, services, replica-sets etc are pretty easy to understand and the alternative for scaling a service/making it discoverable/highly available has a higher barrier of entry IMO. We just don't see it that way because we have lived it. If I told someone that had no experience with dev-ops how to do this in K8s vs setting up load balancers/DNS/health checking it would be much more difficult. The barrier to entry that is currently high (setting up a cluster) is getting much easier with projects and services on many cloud platforms doing that for you.
- tootie 8y ago
  Pods, services and replica sets are not complicated in and of themselves. The problem is that they don't reflect the mental model of a user at all. I just care about instances and load balancers.
  - tybit 8y ago
    OTOH I don’t care about instances and load balancers at all(ideally), I just care about services.
djhworld 8y ago
I have a few raspberry pis at home that I run a few Docker containers on, mostly personal stuff/tools/scripts/servers that I update infrequently but use a lot. Any software I write goes into a container, I like that abstraction. I really want to reduce the friction between developing/maintaing something and deploying/running it. Right now I just have a script for each service that stops the docker container, pulls the latest version from a private repo and starts it again. But that "solution" is getting cumbersome to manage as I add more containers to run and I have to remember what Raspberry Pi each container is running on if I'm running a service that I don't need to have on 2 nodes. I tried to learn Kubernetes a few weeks ago by installing it on one of my Pis, but I quickly realised the learning curve was way beyond what I wanted to be spending my time with (right now - I realise this is a feeble excuse) - especially for home stuff sitting behind a private network. I've been looking at Docker Swarm recently instead which looks a lot simpler. Maybe it will be a good stepping stone to grokking Kubernetes.
- Drdrdrq 8y ago
  A side note, since you mention RPi-s... Do you have any trouble with SD cards (or USB sticks) breaking down on power loss? It seems like a recurring theme and I'm curious how you deal with it in your setup? I am thinking of getting a NAS for my projects instead.
Benjamin_Dobell 8y ago
I must say, I'm presently migrating a client from Dokku to Kubernetes and finding K8s tooling to be severely lacking. I've plenty of experience with Docker and Docker Compose; both for development, and also running on cloud infrastructure (although admittedly not Docker Swarm). I also have some limited experience with Amazon CloudFormation and ECS. I'm no stranger to performing Docker migrations, having previously been responsible for heading up migrations to Docker cloud deployments from "old school" customised on-site rack deployments, VPS LAMPP deployments, and "install everything on this one machine, what's reproducibility?" setups. However, I'm finding that migrating to Kubernetes is by far the worst experience I've had when compared to the alternatives that I've tried. I know cluster deployment/management tends to be a common complaint, however I actually found that to be not too difficult. Thanks in large part to kops. Instead my biggest gripes are: 1. Supposed Immutability Kubernetes treats certain "Dockerisms" (e.g. image tags) as immutable, when they're clearly not. This leads to many subtle "gotchas" and associated work-arounds that are non-intuitive. 2. Templating/substitution (or lack there-of) The argument from the K8s core team seems to be that templating result in non-reproducible deployments. However, I find the opposite to be true. I don't want to maintain slightly different resources for all my different environments and have to manually keep them in sync. That's just asking for trouble. I want a single source of truth. I know Helm (Charts) can be used for templating. However, I find Helm to be over-kill, particularly the Tiller runtime requirement. Helm v3 sounds very promising though, and I have every intention of using Helm more extensively once released. 3. Tooling immaturity / Lack of standardisation I seem to be writing a lot of (simple) bash scripts to solve trivial problems, no doubt others are doing similar things and I'm reinventing the wheel. With CloudFormation/ECS or Docker Compose deployments I don't seem require any scripts at all. Problems can be solved trivially in a declarative way, or by running a single command - of course this approach is supplemented by project-specific documentation. I suppose this largely comes down to K8s flexibility; there's so many different ways to achieve something. I'm not necessarily suggesting stripping away that flexibility, as it's part of what makes K8s great. However, there's a lot that could be simplified for 90% of use-cases, particularly if there were more standardisation and widely agreed upon "best practices". I hope this will come in time. 4. Debugging This isn't at all a fault of K8s itself, just platform immaturity. However, despite the fact K8s seems to be emerging as the winner of the container orchestration war, most alternatives seem to offer significantly better tooling e.g. Docker Compose has proper integration into Jetbrains IDEs, including "remote" debugging of apps running in containers. Jetbrains K8s plugin is currently limited to interpretation of YAML/JSON resource semantics. Don't get me wrong I'm using K8s because I think it's hugely powerful/reliable, and I'm betting on it long-term. Presently, it's truly fantastic from an ops perspective, just significantly less so from a development perspective.
numbsafari 8y ago
I mean... keep in mind that this is a vendor who will make money by convincing you that it's complicated and requires a lot of extra training. So, you gotta take it with a grain of salt. Personally, I've found it to be pretty easy to grasp so far and I'm not the smartest person in the world.

Operyl 8y ago

Super cool! Is only 1 level available right now? EDIT: Further, the front page marketing and actual app don't line up 1:1 very well when it comes to listing lessons.

mstipetic 8y ago
Yeah, sorry, we're still building it up, we didn't plan for this big of a reception, just wanted to test if there's any interest out there
- dfcowell 8y ago
  Good problem to have. :)

mstipetic 8y ago

Hey guys, we're really trying to keep up, but we've gone to 300+ servers in a few minutes. I'm happy to answer any questions here you may have

earth2mars 8y ago
what does that mean? are you going to support more servers or you can't provide more servers?! Thank you for this. looks interesting. would love to get hands-on but it says "We are booting up your cluster!"
- mstipetic 8y ago
  Hey we hit our hard limit of 500 servers, I'm trying to shut down old ones, really sorry about that, but traffic is just crazy
  - nmg 8y ago
    Sounds like the good kind of problem! Let me chime in and say thank you for implementing and sharing this.
    - mstipetic 8y ago
      Thank you very much! Rationally I know this is a good thing, but doesn't feel that way right now
deleted 8y ago
[deleted]
Serow225 8y ago
Hug of death! Congrats :)

mwj 8y ago

How is this different to www.katacoda.com?

vishnu_ks 8y ago
I would also like to know. I initially thought it was katacode.
- mstipetic 8y ago
  we really love katacoda! we're a bit more oriented on trying to visually explain what's happening (not there yet) and we're trying to give you a very high level of interactivity

Arubis 8y ago

Super interested to see how this works once the hug has subsided. This is potentially fortuitous timing; my employer wants me to cover most of this material by end of quarter.

mstipetic 8y ago
Thanks a lot, really didn't expect this kind of traffic. If you ever want to get in touch you can reach me at mislav@<website-domain> or through our slack that's on our landing page and inside console

theyoungwolf 8y ago

Is this for someone who has no experience with any of this or is there prior knowledge required?

mstipetic 8y ago
Hey, we currently host only an intro course but we're building out advanced courses now. So no kubernetes prior knowledge is necessary, but we assume you're a developer with some experience

Drdrdrq 8y ago

> 1 year early bird access > Get 1 year unlimited access to MSB Premium including all lessons and weekly releases. > This is a non-recurring, one time payment. You probably need to fix this, or at least clarify? Is it a yearly sum or one-time payment?

mstipetic 8y ago
Hm, we're doing a presale with unlimited access for 1 year in one payment
- Drdrdrq 8y ago
  That sounds good, but I would suggest clarifying this. Good luck btw!

tjungblut 8y ago

Will it be possible to contribute lessons to your platform? I find conferences a terrible place to share knowledge on K8s, this makes it a lot easier and hands-on.

mstipetic 8y ago
Yes! I'm working on an editor right now, feel free to contact me at mislav at magicsandbox dot com if you're interested

Rafuino 8y ago

Can't find it on your site... where are you spinning up all these servers for the sandbox? GCP? Azure? AWS? Somewhere else?

mstipetic 8y ago
Digital Ocean currently. Just running minikube and kubectl proxy for the feed

StavrosK 8y ago

I found Kubernetes to be easier to learn than the docs would suggest, and I wrote up a short article as a high-level overview. I hope it helps someone, and that it saves you a lot of time: https://www.stavros.io/posts/kubernetes-101/ https://www.stavros.io/posts/kubernetes-101/ You may want to spend two minutes reading this before going into the post's website, as an overview.

lgregg 8y ago

This is awesome, I been looking for something like this.

reactdom 8y ago

If any one is look for Kubernetes books or Kubernetes tutorials: https://reactdom.com/kubernetes https://reactdom.com/kubernetes

alpb 8y ago

What do the Facebook Google Microsoft logos at the bottom of this page mean? Are these companies Magic Sandbox customers?