6 ms·

We managed to run a successful bootcamp school LMS on a single cheapest 1gb ram hetzner vps with rails app, redis cache, postgres, backups, staging env and near

We managed to run a successful bootcamp school LMS on a single cheapest 1gb ram hetzner vps with rails app, redis cache, postgres, backups, staging env and near zero production issues. Not hi loaded of course, but still hundreds active users every single day interacting with the app. Recently had to upgrade to the next tier because of growth. Modern servers are super fast and reliable as long as you know what you’re doing and don’t waste it on unnecessary overheads like k8s etc.

kuschku 4y ago
Even k8s only costs about 10% overhead. It's kind of incredible that "a few hetzner dedis running k8s" still has better reliability than the Cloud™ does.
- capableweb 4y ago
  Remove Kubernetes and "A few Hetzner dedicated servers running just your application" has even better reliability as you remove more points of failures than "A few Hetzner dedicated servers running Kubernetes".
  - carlmr 4y ago
    So much this. System failure is a statistics game. Which is why I like the Erlang approach of having isolated state that you assume will corrupt after a while and you can restart that part of your application with a fresh state. K8s does this, kind of, on a higher level, where the downtime is often more noticeable because there's more to restart. But if your application is already highly fault-tolerant, this is just another point of failure.
    - sammyo 4y ago
      Why hasn't most of the web moved to Erlang (or erlang like) architecture, actually distributed?
      - goodpoint 4y ago
        Because (despite the hype) Erlang embeds principles that can be implemented in other languages and they solve only a narrow set of problems in distributed systems. This does not justify a dedicated programming language.
        bigDinosaur 4y ago
        Is Erlang particularly hyped? I don't usually see it suggested as a solution, I see it almost exclusively in reference to something already written in it, a fait accompli, like WhatsApp.
        carlmr 4y ago
        WhatsApp was scaling really well with a really small team, a good testament to the power of the language. Of course you can take over some of these concepts, but I don't know any other language that is so well designed around the actor model.
      - pdimitar 4y ago
        Rant: Because people adore stable income with no risks, and because most programmers out there are hacks who will only ever learn one or two ancient languages that lost their competitive advantage 10 years ago. Take a look at how insanely far you can go with Elixir + Phoenix on a server with 2 vCPUs and 2GB RAM. It's crazy. At least 80% of all web apps ever written will never get to the point where an upgrade would be needed. So yeah, people love inertia and to parrot the same broken premise quotes like "But StackOverflow shows languages X and Y dominate the industry!". Nah, it doesn't show that at all. It only shows that programmers are people like all others: going out of the comfortable zone is scary so our real industry is manufacturing bullshit post-hoc rationalizations to conceal the fact that we're scared of change. A mega-productive and efficient change at that. Social reasons > objective technical reasons. Always has been true. /rant
        mbrodersen 4y ago
        I completely agree. Most developers simply copy whatever is currently popular without taking into consideration their specific context and biz requirements. Almost all software solutions are massively complexed, with many points of failure, and high latencies for every single transaction, leading to a death spiral of even more complex and expensive “solutions” for “performance” reasons. Microservices is the poster child for this. In contrast, a single high-performance server can handle the full workload of 99.99% of web applications out there. With a separate warm stand-by server ready to take over if needed, and a linear event transaction log as the source of truth.
  - mschuster91 4y ago
    While Kubernetes definitely has its own failure modes and can be an utter pain in the ass to set up, in my experience it also removes significant failure modes and makes your life way easier. For me, the most important thing is I don't have to take care in my deployment pipelines about how many servers I have and how they are named/reachable, to stop Docker containers, to re-create Docker containers, how to deal with logs, letsencrypt cert renewals... all I have to do is point kubectl at the cluster's master, submit the current specification on how the system should look like and I don't have to deal with anything else. If a server or a workload crashes, I don't have to set up monitoring to detect and fail over, Kubernetes will automatically take care of that for me. When I add capacity to the cluster for whatever reason, I enroll the new server with kubeadm and it's online - no entering of new backend IPs in reverse proxies, no bullshit. Maintenance (e.g. OS upgrades, hardware maintenance) becomes a breeze as well: drain the node, do the maintenance, start the node, everything works automatically again.
    - francis-io 4y ago
      As someone who does this day to day, I think k8s is worth it for anything non-trivial. If I didn't already have this understanding, then it might be a different trade off.
- hdjjhhvvhga 4y ago
  > It's kind of incredible It's not incredible, it's normal - it's been always like this.
  - kuschku 4y ago
    I meant it more as in "it's kind of incredible how bad AWS/Azure/GCP is" in terms of reliability and cost ;)
- laputan_machine 4y ago
  Hetzner servers running your code is The Cloud™
  - ihateolives 4y ago
    Hetzner offers bare metal as well. https://www.hetzner.com/sb?hdd_from=0&hdd_to=15500 https://www.hetzner.com/sb?hdd_from=0&hdd_to=15500
    - RealStickman_ 4y ago
      The Cloud is just someone else's computer
capableweb 4y ago
Adding to this: If you use dedicated servers rather than VPS, you can also squeeze out a lot more performance of the boxes as you have dedicated CPUs just for you, instead of shared (vCPU) that others use too.
- yababa_y 4y ago
  AWS and GCP vCPU are absolutely not shared, and any provider that’s still sharing cores or threads is leaving its users open to all manner of atrocious side channels.
  - capableweb 4y ago
    vCPU is a "Virtual Core", it's even in the name. When you rent those, you don't get access to a dedicated core/CPU, but one part of a logical processor. The side channel attacks you mention all have mitigations at this point.
    - yourad_io 4y ago
      A vCPU also represents a CPU _thread_ which is usually half a core. Sometimes you will share the CPU.
    - yababa_y 4y ago
      Only the very smallest instances don’t get a full core. And when you have multiple vCPU the scheduler _won’t_ ever split you onto a thread. This is definitely true of aws’s nitro, I don’t know for certain about gcp. The attacks are not fully mitigated between hyperthreads. Is there evidence cloud compute is more than a few percent slower than on-prem?
      - Sohcahtoa82 4y ago
        > Only the very smallest instances don’t get a full core. Not sure this is true for AWS anymore. Even a t3.nano gives you 2 vCPUs, which I've always interpreted as meaning they give you 1 core with two threads to ensure that you never share a core with another tenant. Of course, t2 instances still exist which give 1 vCPU, but there's no reason to create one of those when a t3 gives 2 vCPUs at a lower price. The only reason to be running a t2 is if you created it before t3 existed and just haven't bothered migrating.
halfstar91 4y ago
If you think Kubernetes is unnecessary overhead, you've never operated at significant scale.
- gtirloni 4y ago
  Exactly. And probably not at significant complexity either.
- throwaway0129u 4y ago
  Not at all, k8s is not designed for very large scale. Unsurprisingly, FAANGS don't use it to manage their own platforms. Edit: Google's Borg is a very different beast. Edit: no need to patronize me. I worked on massive scale deployments otherwise I would not be commenting.
  - halfstar91 4y ago
    Many large organisations use Kubernetes (Google, Spotify, IBM, etc). Regardless, large scale and very large scale are different. Kubernetes is well suited for controlling fleets of compute resource in the order of 10,000s CPU cores, and terabytes of memory. The compute overhead to orchestrate these clusters is well worth the simplicity/standardisation/auto-scaling that comes with Kubernetes. Many people have never had to operate VMs in the hundreds or thousands and do not understand the challenges that come with maintaining varied workloads, alongside capacity planning at that scale.
  - chr15p 4y ago
    a million nodes running a single application is scale, but a thousand nodes running a thousand applications is also scale, and they are very different beasts. The FAANGs operate the first kind, k8s is mostly aimed at the second kind scale, so its designed "for scale", for some definitions of scale.
  - yuppie_scum 4y ago
    K8s spun off of Google’s Borg operator software specifically designed for high availability at FAANG scale. So essentially K8s is the “community edition.” Go read the Google SRE Book for context. We use it to serve ruby with 50 million requests per minute just fine. And the best part is the Horizontal Pod Autoscaler which saves our ass during seasonal spikes. While serverless/lambda are great I do think K8s is the most flexible way to serve rapidly changing containerized workload at scale.
kitd 4y ago
"Unnecessary" in what context?

kuschku 4y ago

Even k8s only costs about 10% overhead. It's kind of incredible that "a few hetzner dedis running k8s" still has better reliability than the Cloud™ does.

capableweb 4y ago
Remove Kubernetes and "A few Hetzner dedicated servers running just your application" has even better reliability as you remove more points of failures than "A few Hetzner dedicated servers running Kubernetes".
- carlmr 4y ago
  So much this. System failure is a statistics game. Which is why I like the Erlang approach of having isolated state that you assume will corrupt after a while and you can restart that part of your application with a fresh state. K8s does this, kind of, on a higher level, where the downtime is often more noticeable because there's more to restart. But if your application is already highly fault-tolerant, this is just another point of failure.
  - sammyo 4y ago
    Why hasn't most of the web moved to Erlang (or erlang like) architecture, actually distributed?
    - goodpoint 4y ago
      Because (despite the hype) Erlang embeds principles that can be implemented in other languages and they solve only a narrow set of problems in distributed systems. This does not justify a dedicated programming language.
      - bigDinosaur 4y ago
        Is Erlang particularly hyped? I don't usually see it suggested as a solution, I see it almost exclusively in reference to something already written in it, a fait accompli, like WhatsApp.
        carlmr 4y ago
        WhatsApp was scaling really well with a really small team, a good testament to the power of the language. Of course you can take over some of these concepts, but I don't know any other language that is so well designed around the actor model.
    - pdimitar 4y ago
      Rant: Because people adore stable income with no risks, and because most programmers out there are hacks who will only ever learn one or two ancient languages that lost their competitive advantage 10 years ago. Take a look at how insanely far you can go with Elixir + Phoenix on a server with 2 vCPUs and 2GB RAM. It's crazy. At least 80% of all web apps ever written will never get to the point where an upgrade would be needed. So yeah, people love inertia and to parrot the same broken premise quotes like "But StackOverflow shows languages X and Y dominate the industry!". Nah, it doesn't show that at all. It only shows that programmers are people like all others: going out of the comfortable zone is scary so our real industry is manufacturing bullshit post-hoc rationalizations to conceal the fact that we're scared of change. A mega-productive and efficient change at that. Social reasons > objective technical reasons. Always has been true. /rant
      - mbrodersen 4y ago
        I completely agree. Most developers simply copy whatever is currently popular without taking into consideration their specific context and biz requirements. Almost all software solutions are massively complexed, with many points of failure, and high latencies for every single transaction, leading to a death spiral of even more complex and expensive “solutions” for “performance” reasons. Microservices is the poster child for this. In contrast, a single high-performance server can handle the full workload of 99.99% of web applications out there. With a separate warm stand-by server ready to take over if needed, and a linear event transaction log as the source of truth.
- mschuster91 4y ago
  While Kubernetes definitely has its own failure modes and can be an utter pain in the ass to set up, in my experience it also removes significant failure modes and makes your life way easier. For me, the most important thing is I don't have to take care in my deployment pipelines about how many servers I have and how they are named/reachable, to stop Docker containers, to re-create Docker containers, how to deal with logs, letsencrypt cert renewals... all I have to do is point kubectl at the cluster's master, submit the current specification on how the system should look like and I don't have to deal with anything else. If a server or a workload crashes, I don't have to set up monitoring to detect and fail over, Kubernetes will automatically take care of that for me. When I add capacity to the cluster for whatever reason, I enroll the new server with kubeadm and it's online - no entering of new backend IPs in reverse proxies, no bullshit. Maintenance (e.g. OS upgrades, hardware maintenance) becomes a breeze as well: drain the node, do the maintenance, start the node, everything works automatically again.
  - francis-io 4y ago
    As someone who does this day to day, I think k8s is worth it for anything non-trivial. If I didn't already have this understanding, then it might be a different trade off.
hdjjhhvvhga 4y ago
> It's kind of incredible It's not incredible, it's normal - it's been always like this.
- kuschku 4y ago
  I meant it more as in "it's kind of incredible how bad AWS/Azure/GCP is" in terms of reliability and cost ;)
laputan_machine 4y ago
Hetzner servers running your code is The Cloud™
- ihateolives 4y ago
  Hetzner offers bare metal as well. https://www.hetzner.com/sb?hdd_from=0&hdd_to=15500 https://www.hetzner.com/sb?hdd_from=0&hdd_to=15500
  - RealStickman_ 4y ago
    The Cloud is just someone else's computer

capableweb 4y ago

Adding to this: If you use dedicated servers rather than VPS, you can also squeeze out a lot more performance of the boxes as you have dedicated CPUs just for you, instead of shared (vCPU) that others use too.

yababa_y 4y ago
AWS and GCP vCPU are absolutely not shared, and any provider that’s still sharing cores or threads is leaving its users open to all manner of atrocious side channels.
- capableweb 4y ago
  vCPU is a "Virtual Core", it's even in the name. When you rent those, you don't get access to a dedicated core/CPU, but one part of a logical processor. The side channel attacks you mention all have mitigations at this point.
  - yourad_io 4y ago
    A vCPU also represents a CPU _thread_ which is usually half a core. Sometimes you will share the CPU.
  - yababa_y 4y ago
    Only the very smallest instances don’t get a full core. And when you have multiple vCPU the scheduler _won’t_ ever split you onto a thread. This is definitely true of aws’s nitro, I don’t know for certain about gcp. The attacks are not fully mitigated between hyperthreads. Is there evidence cloud compute is more than a few percent slower than on-prem?
    - Sohcahtoa82 4y ago
      > Only the very smallest instances don’t get a full core. Not sure this is true for AWS anymore. Even a t3.nano gives you 2 vCPUs, which I've always interpreted as meaning they give you 1 core with two threads to ensure that you never share a core with another tenant. Of course, t2 instances still exist which give 1 vCPU, but there's no reason to create one of those when a t3 gives 2 vCPUs at a lower price. The only reason to be running a t2 is if you created it before t3 existed and just haven't bothered migrating.

halfstar91 4y ago

If you think Kubernetes is unnecessary overhead, you've never operated at significant scale.

gtirloni 4y ago
Exactly. And probably not at significant complexity either.
throwaway0129u 4y ago
Not at all, k8s is not designed for very large scale. Unsurprisingly, FAANGS don't use it to manage their own platforms. Edit: Google's Borg is a very different beast. Edit: no need to patronize me. I worked on massive scale deployments otherwise I would not be commenting.
- halfstar91 4y ago
  Many large organisations use Kubernetes (Google, Spotify, IBM, etc). Regardless, large scale and very large scale are different. Kubernetes is well suited for controlling fleets of compute resource in the order of 10,000s CPU cores, and terabytes of memory. The compute overhead to orchestrate these clusters is well worth the simplicity/standardisation/auto-scaling that comes with Kubernetes. Many people have never had to operate VMs in the hundreds or thousands and do not understand the challenges that come with maintaining varied workloads, alongside capacity planning at that scale.
- chr15p 4y ago
  a million nodes running a single application is scale, but a thousand nodes running a thousand applications is also scale, and they are very different beasts. The FAANGs operate the first kind, k8s is mostly aimed at the second kind scale, so its designed "for scale", for some definitions of scale.
- yuppie_scum 4y ago
  K8s spun off of Google’s Borg operator software specifically designed for high availability at FAANG scale. So essentially K8s is the “community edition.” Go read the Google SRE Book for context. We use it to serve ruby with 50 million requests per minute just fine. And the best part is the Horizontal Pod Autoscaler which saves our ass during seasonal spikes. While serverless/lambda are great I do think K8s is the most flexible way to serve rapidly changing containerized workload at scale.

kitd 4y ago

"Unnecessary" in what context?