5 ms·

Advice on moving to the cloud?

18 points by niels 16y ago

wccrawford 16y ago
I'll save everyone some reading. The advice was: Don't do it. While I agree that in their particular circumstances that it's probably the correct answer, I don't think it's right for everyone. For one thing, their system isn't tolerant enough of downtime. If one of their databases goes offline, they've got some manual fixing to do to get it back up and going. Ugh. (I took this to mean that if you unplug the network cable, they have manual fixing to do. They might have meant power loss... And that's a little more understandable, but still not something you really want in the cloud.)
- schammy 16y ago
  I was referring to power loss. Although in the cloud, it's not actual "power loss" that's the issue, but the end result is the same - the database is shut down uncleanly and must be dealt with before coming back online.
thehodge 16y ago
I think more interesting than the question is the fact that the questions was asked at all, by a company to the customers. I'm a customer of getclickys and in most cases openness like this would be refreshing and it is but there is an element of 'flying by the seat of our pants' that could come across with this post.
- schammy 16y ago
  Yes, someone on our blog raised a similar point. The reason we are asking for advice is because we have zero experience running an app in a cloud environment. Well, we did build our own custom CDN using DynDNS and vps.net, but that's just a small part of our service. I'm scared of the cloud for running everything, so just asking for anyone with experience to tell us what they think of the idea.
cparedes 16y ago
You'll have to design your stuff to tolerate failure; use things like Chef for quick deployments so that provisioning new servers isn't a huge pain in the ass. Further, the biggest thing to keep in mind is that your resources are shared. This is not too big of a deal for some applications, but for a database server, this can spell certain death (I/O doesn't virtualize too well; you can mitigate some of this by deploying a lot of slave machines for your reads.)
KevinMS 16y ago
I have very very limited experience with EC2, a single small instance, for over a month, but I think that gives me more credibility when I say this, don't use EC2 unless you designed your whole app to run on servers you expect to catch fire and explode at any time, because my single little instance died within a month, then twice it, well, did something I've never seen happen before, blow out a device driver? So I couldn't ssh in, all I got was "PTY allocation request failed on channel 0", so now I'm on my fourth instance. I couldn't imagine the carnage if I had been running tens or hundreds of instances.
- schammy 16y ago
  Thanks. That is exactly my fear. Most of the commenters on our blog told us this probably isn't a good idea so we're very unlikely to do it at this point. I'M JUST SO DAMN SICK OF DEALING WITH PHYSICAL SERVERS. :)
  - KevinMS 16y ago
    There's also the option of splitting your app between EC2 and your own servers, so you can run the more ephemeral stuff like web servers and caching in the cloud and the bedrock stuff like databases on your own quality hardware. Then you'll have less physical servers to deal with :)
    - schammy 16y ago
      My concern about that is communication between web and DB servers. Some reports need hundreds of queries to generate - wouldn't the lag severely slow this down? Most of our servers are DB anyways. We have 2 load balancers, 6 web servers, and ~40 DB servers.
      - sokoloff 16y ago
        Yes, it would, but you might be better served to rethink your app design to return multiple result sets, and/or do most of the relevant joining/filtering on the database, even if your DB is <1ms away from your web farm. We run a good sized e-commerce site (Alexa top 1000; barely) and our ratio of DB queries per dynamic page is right around 2.5:1. You might not be able to get your query-per-page ratio into that range, but hundreds is certainly too many, IMO. Even "dozens" would be cause for concern, IMO. Moving your webfarm to a different datacenter than your DB is going to shine a spotlight on that problem, but even without that spotlight, you still have a problem. PS: Even with pushing that amount of work to the DB, we have almost the inverted ratio of web servers to DB servers as you report.
benologist 16y ago
I have a very similar volume site (game analytics) ... I've always been afraid of Amazon because of the bandwidth and request based billing. I just use VPSs to scale when I have to ... it helps that my hosting guy is on my MSN Messenger. Edit: Also your site looks terrible in Opera ... the CSS or JS or something isn't loading which is breaking everything.
- FluidDjango 16y ago
  FWIW: I'm not seeing any such phenom. My Opera (10.62) is scarcely discernible from Safari - both of which look very attractive.
  - benologist 16y ago
    Oooh I know what it was. One of the tech blogs had some annoying shit happening when you right clicked URLs that made the right click menu not show for me (slo0w connection here) so I blocked the subdomain.

18 points by niels 16y ago

I'll save everyone some reading. The advice was: Don't do it. While I agree that in their particular circumstances that it's probably the correct answer, I don't think it's right for everyone. For one thing, their system isn't tolerant enough of downtime. If one of their databases goes offline, they've got some manual fixing to do to get it back up and going. Ugh. (I took this to mean that if you unplug the network cable, they have manual fixing to do. They might have meant power loss... And that's a little more understandable, but still not something you really want in the cloud.)

schammy 16y ago
I was referring to power loss. Although in the cloud, it's not actual "power loss" that's the issue, but the end result is the same - the database is shut down uncleanly and must be dealt with before coming back online.

thehodge 16y ago

I think more interesting than the question is the fact that the questions was asked at all, by a company to the customers. I'm a customer of getclickys and in most cases openness like this would be refreshing and it is but there is an element of 'flying by the seat of our pants' that could come across with this post.

schammy 16y ago
Yes, someone on our blog raised a similar point. The reason we are asking for advice is because we have zero experience running an app in a cloud environment. Well, we did build our own custom CDN using DynDNS and vps.net, but that's just a small part of our service. I'm scared of the cloud for running everything, so just asking for anyone with experience to tell us what they think of the idea.

cparedes 16y ago

You'll have to design your stuff to tolerate failure; use things like Chef for quick deployments so that provisioning new servers isn't a huge pain in the ass. Further, the biggest thing to keep in mind is that your resources are shared. This is not too big of a deal for some applications, but for a database server, this can spell certain death (I/O doesn't virtualize too well; you can mitigate some of this by deploying a lot of slave machines for your reads.)

KevinMS 16y ago

I have very very limited experience with EC2, a single small instance, for over a month, but I think that gives me more credibility when I say this, don't use EC2 unless you designed your whole app to run on servers you expect to catch fire and explode at any time, because my single little instance died within a month, then twice it, well, did something I've never seen happen before, blow out a device driver? So I couldn't ssh in, all I got was "PTY allocation request failed on channel 0", so now I'm on my fourth instance. I couldn't imagine the carnage if I had been running tens or hundreds of instances.

schammy 16y ago
Thanks. That is exactly my fear. Most of the commenters on our blog told us this probably isn't a good idea so we're very unlikely to do it at this point. I'M JUST SO DAMN SICK OF DEALING WITH PHYSICAL SERVERS. :)
- KevinMS 16y ago
  There's also the option of splitting your app between EC2 and your own servers, so you can run the more ephemeral stuff like web servers and caching in the cloud and the bedrock stuff like databases on your own quality hardware. Then you'll have less physical servers to deal with :)
  - schammy 16y ago
    My concern about that is communication between web and DB servers. Some reports need hundreds of queries to generate - wouldn't the lag severely slow this down? Most of our servers are DB anyways. We have 2 load balancers, 6 web servers, and ~40 DB servers.
    - sokoloff 16y ago
      Yes, it would, but you might be better served to rethink your app design to return multiple result sets, and/or do most of the relevant joining/filtering on the database, even if your DB is <1ms away from your web farm. We run a good sized e-commerce site (Alexa top 1000; barely) and our ratio of DB queries per dynamic page is right around 2.5:1. You might not be able to get your query-per-page ratio into that range, but hundreds is certainly too many, IMO. Even "dozens" would be cause for concern, IMO. Moving your webfarm to a different datacenter than your DB is going to shine a spotlight on that problem, but even without that spotlight, you still have a problem. PS: Even with pushing that amount of work to the DB, we have almost the inverted ratio of web servers to DB servers as you report.

benologist 16y ago

I have a very similar volume site (game analytics) ... I've always been afraid of Amazon because of the bandwidth and request based billing. I just use VPSs to scale when I have to ... it helps that my hosting guy is on my MSN Messenger. Edit: Also your site looks terrible in Opera ... the CSS or JS or something isn't loading which is breaking everything.

FluidDjango 16y ago
FWIW: I'm not seeing any such phenom. My Opera (10.62) is scarcely discernible from Safari - both of which look very attractive.
- benologist 16y ago
  Oooh I know what it was. One of the tech blogs had some annoying shit happening when you right clicked URLs that made the right click menu not show for me (slo0w connection here) so I blocked the subdomain.