4 ms·

Ah, really? We are moved from Google Cloud, we just needed one machine with average IOPS to store our files (50-100gb only), but we are constantly hitting limit

by ex3ndr 10y ago

Ah, really? We are moved from Google Cloud, we just needed one machine with average IOPS to store our files (50-100gb only), but we are constantly hitting limits of IOPS, file upload for our users at some point became a nightmare. What's a problem with cloud when we pay for it 1k$+? Renting one server costs ~50$/mo and will cover everything. Only thing is needed is some adjustments in configuration. But, thanks to k8s we can roll out servers in a week. I am wonder now why we was needed to sit on clouds when everything works 10x times slower at 10x price?

static_noise 10y ago
What if your hardware fails? What if you need to scale up (or scale down again)? Making servers run as reliable as in cloud datacenters is really hard work and imposes additional cost. I.e. you need not one, but two or three datacenters with 2-3 times the amount of servers you actually use. Then you need admin staff who knows their work and keeps things running smoothly and reliably. You need not one but at least 3 of that staff because they might get sick or go on vacation. Your cost savings probably comes from cutting corners. Depending on the structure of your applications, downtime requirements and recovery plans, that may even be OK. You don't always need a fleet of tanks to deliver a box of milk bottles.
- ex3ndr 10y ago
  Hardware fails? It will start on other node automagically. May be we will need to switch psql instance. What kind of additional cost? Do you ever hit something really awful with k8s on self-hosted servers? What can go wrong except losing node?
  - static_noise 10y ago
    So, you have redundancy and failover at the application level. You don't need reliable servers then.
    - empthought 10y ago
      Reliability is with respect to some SLA; it could very well be that one would _like_ to timeout more quickly than can be reasonably expected of cloud infrastructure, but well within the capabilities of hardware itself.
- ZenoArrow 10y ago
  > "Making servers run as reliable as in cloud datacenters is really hard work and imposes additional cost." Not really. It's easy enough to have dedicated server hardware that runs just as smoothly as cloud hosted hardware. There are many options for doing so. The key is to pay for support. For example, if you go with Microsoft or Red Hat or Ubuntu servers, you can pay for support contracts to get help from experts in configuring systems. Such options can work out cheaper than cloud hosting (depending on the IT infrastructure requirements), with the added benefit of having hardware more directly under your control.
  - Pyxl101 10y ago
    What happens when you run into trouble with one of the servers? Like it's suddenly stopped responding and you can't SSH? For my systems built on EC2, I might not need to do anything at all for typical hardware failures, if I've set up EC2 auto recovery. It transparently relaunches my instance on another server. If I'm not using auto recovery, then I might just stop and restart the instance, in which case I'm also migrated to another physical server if the first one wasn't working. It has the same identity, by the way: same IP address, same disk contents, and all that, and all the machine perceived was an OS reboot (or maybe kernel panic followed by reboot, depending on the failure). For stateless systems I don't even need to bother with that. I'll just configure it to spin up a replacement instance if the desired number of servers I want to have online is not met because one of them failed. When I have 100+ servers in a fleet, this is really convenient. I don't need to keep track of them individually, I just say: "This is the image I'd like to run, and I want to have 100 of them". Hardware failures that would bring down a normal physical machine don't need to involve me at all. The old server hardware that's broken is now in the hands of EC2 to repair or swap out, and my server's back up and running in minutes, possibly with no action on my part. Is something like that easy to achieve with colocation and support offerings with those vendors? Genuine question - I've never operated in colo or worked with those vendors. I realize that colo facilities can take care of repairing and swapping servers for me, but the benefit I get from the cloud is that I can perform those actions in moments with an API, or they can happen automatically, and I don't need to engage other humans. I can't imagine spending my time coordinating with vendors, getting on the phone, opening support tickets, waiting for people to do things, etc... right now I can do everything at the speed of computing with APIs and automation. It'd be tough to give that up.
- user5994461 10y ago
  > You don't always need a fleet of tanks to deliver a box of milk bottles. A bottle of milk filled with Gold is worth about $800k. By extension the 6-pack is $4.8M If tanks can fit seamlessly in the budget, we shall give tanks a serious thoughts! :D
user5994461 10y ago
If your setup can run on a $50 server, you shouldn't use AWS/GCE ^^ Out of curiosity. What were you running? What was your setup? How much disk IOPS? and disk bandwidth used?
- ex3ndr 10y ago
  Are you sure that (just example) - https://www.hetzner.de/us/hosting/produkte_rootserver/ex41sssd https://www.hetzner.de/us/hosting/produkte_rootserver/ex41ss... is a weak server? Running same VM on AWS/GCE will cost 10x more.
  - user5994461 10y ago
    OMG A server WITHOUT ECC memory. #fear Let's not call that a server but a desktop computer please ^^ --- The point of the cloud is to manage MANY servers. If all you have is a single box, you don't need help to manage it, you don't need AWS/GCE.
    - ex3ndr 10y ago
      If i can buy two desktops instead of one server then i can made a nice failover configuration instead of using just one server. Also as i know google and fb think that ecc is useless, isn't it? Well, just order 50 of them and install k8s.
      - puzzle 10y ago
        Failover is useless when you have already increased your chances of corrupting data. Even with ECC there's some degree of corruption; doing without is suicidal. Google figured quite early in its life that RAM was the one component not to skimp on: http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html

by ex3ndr 10y ago

static_noise 10y ago

What if your hardware fails? What if you need to scale up (or scale down again)? Making servers run as reliable as in cloud datacenters is really hard work and imposes additional cost. I.e. you need not one, but two or three datacenters with 2-3 times the amount of servers you actually use. Then you need admin staff who knows their work and keeps things running smoothly and reliably. You need not one but at least 3 of that staff because they might get sick or go on vacation. Your cost savings probably comes from cutting corners. Depending on the structure of your applications, downtime requirements and recovery plans, that may even be OK. You don't always need a fleet of tanks to deliver a box of milk bottles.

ex3ndr 10y ago
Hardware fails? It will start on other node automagically. May be we will need to switch psql instance. What kind of additional cost? Do you ever hit something really awful with k8s on self-hosted servers? What can go wrong except losing node?
- static_noise 10y ago
  So, you have redundancy and failover at the application level. You don't need reliable servers then.
  - empthought 10y ago
    Reliability is with respect to some SLA; it could very well be that one would _like_ to timeout more quickly than can be reasonably expected of cloud infrastructure, but well within the capabilities of hardware itself.
ZenoArrow 10y ago
> "Making servers run as reliable as in cloud datacenters is really hard work and imposes additional cost." Not really. It's easy enough to have dedicated server hardware that runs just as smoothly as cloud hosted hardware. There are many options for doing so. The key is to pay for support. For example, if you go with Microsoft or Red Hat or Ubuntu servers, you can pay for support contracts to get help from experts in configuring systems. Such options can work out cheaper than cloud hosting (depending on the IT infrastructure requirements), with the added benefit of having hardware more directly under your control.
- Pyxl101 10y ago
  What happens when you run into trouble with one of the servers? Like it's suddenly stopped responding and you can't SSH? For my systems built on EC2, I might not need to do anything at all for typical hardware failures, if I've set up EC2 auto recovery. It transparently relaunches my instance on another server. If I'm not using auto recovery, then I might just stop and restart the instance, in which case I'm also migrated to another physical server if the first one wasn't working. It has the same identity, by the way: same IP address, same disk contents, and all that, and all the machine perceived was an OS reboot (or maybe kernel panic followed by reboot, depending on the failure). For stateless systems I don't even need to bother with that. I'll just configure it to spin up a replacement instance if the desired number of servers I want to have online is not met because one of them failed. When I have 100+ servers in a fleet, this is really convenient. I don't need to keep track of them individually, I just say: "This is the image I'd like to run, and I want to have 100 of them". Hardware failures that would bring down a normal physical machine don't need to involve me at all. The old server hardware that's broken is now in the hands of EC2 to repair or swap out, and my server's back up and running in minutes, possibly with no action on my part. Is something like that easy to achieve with colocation and support offerings with those vendors? Genuine question - I've never operated in colo or worked with those vendors. I realize that colo facilities can take care of repairing and swapping servers for me, but the benefit I get from the cloud is that I can perform those actions in moments with an API, or they can happen automatically, and I don't need to engage other humans. I can't imagine spending my time coordinating with vendors, getting on the phone, opening support tickets, waiting for people to do things, etc... right now I can do everything at the speed of computing with APIs and automation. It'd be tough to give that up.
user5994461 10y ago
> You don't always need a fleet of tanks to deliver a box of milk bottles. A bottle of milk filled with Gold is worth about $800k. By extension the 6-pack is $4.8M If tanks can fit seamlessly in the budget, we shall give tanks a serious thoughts! :D

user5994461 10y ago

If your setup can run on a $50 server, you shouldn't use AWS/GCE ^^ Out of curiosity. What were you running? What was your setup? How much disk IOPS? and disk bandwidth used?

ex3ndr 10y ago
Are you sure that (just example) - https://www.hetzner.de/us/hosting/produkte_rootserver/ex41sssd https://www.hetzner.de/us/hosting/produkte_rootserver/ex41ss... is a weak server? Running same VM on AWS/GCE will cost 10x more.
- user5994461 10y ago
  OMG A server WITHOUT ECC memory. #fear Let's not call that a server but a desktop computer please ^^ --- The point of the cloud is to manage MANY servers. If all you have is a single box, you don't need help to manage it, you don't need AWS/GCE.
  - ex3ndr 10y ago
    If i can buy two desktops instead of one server then i can made a nice failover configuration instead of using just one server. Also as i know google and fb think that ecc is useless, isn't it? Well, just order 50 of them and install k8s.
    - puzzle 10y ago
      Failover is useless when you have already increased your chances of corrupting data. Even with ECC there's some degree of corruption; doing without is suicidal. Google figured quite early in its life that RAM was the one component not to skimp on: http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html