14 ms·

We're moving. Goodbye Rackspace.

203 points by suhail 16y ago

powdahound 16y ago
Goodbye code.mixpanel.com too? Page won't load. :(
- suhail 16y ago
  funny part is...it's on the rackspace cloud. Though probably our fault since we gave it a crappy box. Edit: 512MB of RAM should be enough...
  - riffer 16y ago
    What is the limiting factor here? I am curious because it seems like almost any configuration & setup should be able to serve a blog post to the volume of request that come with being at the top of HN.
  - suhail 16y ago
    Apache2 basically just OOM'd
    - madaxe 16y ago
      You misconfigured it, then, I'm afraid to say... Number of threads = server memory/maximum memory usage per thread - 10% margin for overheads.
    - chaosmachine 16y ago
      Disable keepalives: http://www.kalzumeus.com/2010/06/19/running-apache-on-a-memory-constrained-vps/ http://www.kalzumeus.com/2010/06/19/running-apache-on-a-memo...
- redorb 16y ago
  google-cache-version http://webcache.googleusercontent.com/search?sourceid=chrome&ie=UTF-8&q=cache:code.mixpanel.com/amazon-vs-rackspace/ http://webcache.googleusercontent.com/search?sourceid=chrome...
- seldo 16y ago
  Yeah, not exactly a ringing endorsement of rackspace that the rackspace-hosted blog saying they're leaving is down under the load.
barrydahlberg 16y ago
I'm suprised to see a setup using 50+ instances running on RackSpace cloud, wouldn't it have made sense to start moving towards dedicated servers by then?
- cperciva 16y ago
  Not necessarily. Sometimes flexibility is worth more than having the minimum possible cost per clock cycle.
  - barrydahlberg 16y ago
    True but in theory the advantage with RackSpace is you can mix and match to balance the cost of your known requirements with the flexiblity for your unknown.
- suhail 16y ago
  We need the elasticity. I can't wait 24 hours for new boxes to come.
  - tnm 16y ago
    Capacity planning. It's a thing. Try it out.
    - deleted 16y ago
      [deleted]
    - suhail 16y ago
      Capacity planning is hard when data volume doubles / triples over night.
      - jamesgolick 16y ago
        The fundamentals of capacity planning do not change based on the magnitude of your data growth. Why would they? We're mostly talking about looking at your data growth curve and extrapolating points in the future. Why would that become impossible just because the curve is steep?
powdahound 16y ago
Make sure you explore the limits of EBS before assuming it's a perfect solution. We've found it to have incredibly slow throughput at times. Some reference links: - http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs - http://victortrac.com/EC2_Ephemeral_Disks_vs_EBS_Volumes - http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html
- gyardley 16y ago
  We had these issues as well. I'd certainly try AWS, but I'd wait a couple of months before shelling out the upfront cost for the Reserved Instances.
  - blantonl 16y ago
    EBS definitely has performance limitations, however creating striped RAID arrays of EBS volumes significantly helps with performance. This blog post walks through a bunch of things you can do to make the best out of the flexibility you get with EBS, and includes scheduler tuning, RAID block size planning, etc. http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs/ http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs... My team runs a large AWS/EBS backed implementation and, while we've seen some of the EBS performance issues, we've certainly engineered past them to get us where we need to be.
    - chrisbolt 16y ago
      Wouldn't creating striped RAID arrays of EBS volumes add to the cost and remove the ability to do snapshots? How do you get a consistent snapshot of multiple striped EBS volumes?
      - spudlyo 16y ago
        It just makes it more complicated. You have lock and snap all of the volumes at once. The ec2-consistent-snapshot tool makes this easier to manage.
      - 16y ago
cperciva 16y ago
Quoth the article: Lastly, we moved over to the Rackspace Cloud because they cut a deal with YCombinator (one of the many benefits of being part of YC). Can anyone say what this deal is, or is it secret?
- varenc 16y ago
  Just reduced price hosting. My guess is its a deal that won't be happening again? :-)
- mikeyur 16y ago
  For a separate project I've worked on (non-YC), we negotiated 50% off bandwidth and somewhat reduced server rates. You just need to ask for it. Not sure what the YC deal is, but I assume it's similar.
  - aaroneous 16y ago
    I had a similar experience: with minimal negotiating we were able to get 50% off their initial quote. I also renegotiated our terms later within the contract period with relative ease. They pad their pricing pretty heavily, so don't be afraid to ask.
- mattlong 16y ago
  Can't speak for other YC companies (it may vary from batch to batch), but for us it was a generous rackspace cloud monthly credit for 12 months.
joecode 16y ago
One potential reason not to move: Security. A good friend deep in the security community once told me, off hand, that EC2 was "owned." I didn't take this too seriously until another good friend, who has been working at Amazon for the past several years, told me that engineers at Amazon were generally forbidden from using AWS due to security concerns. That much said, I still decided to use EC2/RDS/S3 to host the infrastructure of my latest startup. It is just too convenient to walk away from. Once it matters, I can move the critical stuff to dedicated servers. EDIT: To clarify, I'm not suggesting that Amazon knows AWS is "owned" and offers it to others anyway. I'm only noting that, for certain critical services, they themselves do not appear willing to take the risk.
- rbranson 16y ago
  While this is disconcerting, I wouldn't make any business decisions based on such a claim. The idea that EC2 is "owned" without Amazon knowing about it is closing in on absurdity. I've worked directly with Amazon as an outside vendor and they are very security concious, to the point of near paranoia.
  - joecode 16y ago
    While I agree it is hard to believe, it would be even more surprising if Amazon did know about it. The fact that they do not use AWS internally suggests that---at least with their level of paranoia---they seem to suspect AWS themselves.
    - benblack 16y ago
      Both you and your friend are misinformed.
      - joecode 16y ago
        I'm not sure how you can say that so matter-of-factly. My security friend was talking about something that Amazon does not (and presumably very few people do) know. Meanwhile my friend at Amazon was just stating the fact that he was not supposed to use AWS, or only with extreme caution. Of course that may differ from department to department, if that's what you mean.
yesimahuman 16y ago
Another control panel complaint: DNS. For some reason beyond me you have to choose a server just to configure DNS. Added on top of the fact that the control panel is really slow, it just becomes a pain to use.
- nl 16y ago
  Yes!! I was trying to do a DNS change and I couldn't find the DNS panel anywhere until I Google'd it.
jread 16y ago
CPU is a major bottleneck for Rackspace Cloud. All instance sizes get the same 4 cores and about the same compute resources. CPU performance is roughly the same on a 1GB cloud server as an 8GB cloud server, you are just paying for more memory. Rackspace also uses ONLY Opteron 2374 2.20 GHz processors. EC2 on the other hand offers linear CPU performance improvement on larger sized instances. EC2 also uses a heterogenous hardware ranging from Opteron 2218 or Xeon E5430 for m1 instances; Xeon E5410 for c1 instances; Xeon X5550 for m2 instances and Xeon X5570 (hyper-threaded to 16 cores) for the cluster compute instances. EBS on the cluster instance is also much faster than local disk IO in the Rackspace Cloud based on testing I've done (due to non-blocking 10G network). Here are a couple of references for this: http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchmarking-in-cloud.html http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma... http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in-cloud.html http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in...
jread 16y ago
<quote>Amazon has a CDN and servers distributed globally. This is important to Mixpanel as websites all over the world are sending us data. There’s nothing like this on Rackspace.</quote> Actually, Rackspace cloud offers CDN services with Cloud Files through Limelight, although it does not support some features that CloudFront does like CName and streaming.
- suhail 16y ago
  But not physical servers which is really what I am referring to.
  - jread 16y ago
    Rackspace Cloud has data centers in Texas and Illinois. However, they don't let you choose (your account is assigned to one data center at the time of creation). EC2 certain offers a lot more flexibility in this regard with EC2 data center regions in California, Virginia, Ireland and Singapore. I spoke with Rackspace at Interop a few weeks ago and they told me they are working on expanding to additional data centers including an international data center very soon and offering choice.
    - 0x44 16y ago
      You can "choose" insofar as you can email support and they can manually change what data-center your next VM and all subsequent ones will be built in. It really should be exposed through the API, but the people who write the customer-facing API haven't done it.
      - jread 16y ago
        That is not the story I got when I contacted support and asked if I could deploy a VM specifically to the Chicago cloud. Rackspace's support told me this was not possible an I'd have to setup an entirely new account in order to deploy to that data center.
        16y ago
adriand 16y ago
This is a rather unpleasant review of a service we were about to move over to. For companies that have moderate performance requirements (e.g. visitors in the range of 30k or 40k per day across a range of web apps and sites), reasonable but by no means expert level server administration skills, the need for a redundant environment to satisfy SLAs with clients (e.g. two app servers + load balancer + master/slave db servers), and the desire to focus mainly on software development instead of server admin, what companies does the HN community recommend? We've been considering Rackspace Cloud and Linode, but are open to any suggestions. We also have a quote for a standard, managed four-server + hardware load balancer deployment in front of us but it is pricey ($3000+/month).
- sanswork 16y ago
  I've used Rackspace, Amazon and GoGrid. I'm currently almost entirely on GoGrid and just finishing up migrating back off of Amazon for the second time(Migrated some services back to them when they opened west coast). There have been some stability issues in the past year(instances going down for maintenance regularly) but haven't had a problem with that in a while and was advised it was due to them upgrading all the servers in their cluster. The ability to mix dedicated and cloud systems easily has been the killer feature for me. We also prepay and the number of servers we get for the cost is pretty good.
- madaxe 16y ago
  Linode are wonderful - best support I've had from anyone, ever. Nice instances, too!
  - shazow 16y ago
    +1 to Linode.
  - catch23 16y ago
    another +1. we moved to linode from rackspace. They don't have that crazy backup limitation that rackspace does, and they're always open to custom inquiries. Rackspace has the big company mentality -- if our interface doesn't have an option for it, it doesn't exist.
newobj 16y ago
Wow, that's a brutal under-the-bus-tossing. But well deserved. Most of the points are spot-on. I've operated a several dozen machine fleet for 2+ years on EC2 and I can tell you that the number of times boxes have gone down never to come back (which most people tend to think happens regularly on EC2) is incredibly small. We actually run a Hadoop cluster (including DFS) on a spot instances. We never lose the spot bid. We pay way less than the going rate for the compute time. Less than the reserved instance rate. It's awesome. Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye. It's not too hard. I would second another commenter's caution on EBS though. We never put it into production. Personally I never experienced an ephemeral drive failure that had repercussions - when they (rarely) occurred, the drive was RAIDed or in our Hadoop cluster (e.g. redundant). We made two experiments into using EBS with our DB's, and both times, literally within 24 hours, we experienced a catastrophic failure of the EBS volume, one time unrecoverable. So that put me pretty well off EBS. I can't say that the performance is necessarily the best, and we do experience the occasional odd asymmetrical inter-machine latencies (e.g. 300ms to establish a tcp connection in one direction, but normal <1ms in the other), but for the most part AWS is just awesome.
- gruseom 16y ago
  Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye. How long do the spot instances typically run before they vanish?
  - cperciva 16y ago
    A spot instance should stay around until the auction price is higher than what you bid. That price has been quite stable recently, so as long as you bid slightly higher than that rate you'll probably have your instance for months.
    - gruseom 16y ago
      That's better than I was expecting. I'm surprised more people don't take advantage of this (or maybe they do, I don't know). Use S3 for permanent storage, ephemeral disk and RAM for cache, and pay spot rates. Sounds pretty good if one's design allows for it.
dangrover 16y ago
Rackspace cloud's DNS stuff stinks. No way to add TXT records -- you have to open a ticket! Sure you can host it yourself, but every other cloud provider has this in their UI. I get the feeling they're just in "maintenence mode" over there and don't have anyone working hard on improving the offerings.
- danudey 16y ago
  Their DNS used to be great, if simple. You could put in a record for your new server and by the time you'd logged in, the record was live. Now, it's gotten to the point where their DNS updates so slowly that I just re-use old subdomains instead of creating new ones, just so that I can get my project tested today rather than tomorrow. I'm working on moving everything over to Amazon; more flexibility. I like Rackspace Cloud, but they're falling behind and they either don't know, don't care, or can't catch up.
- megaman821 16y ago
  What is even more sad, is that their DNS console for hosted servers and Slicehost is so much better. They are literally sitting on something better and giving their cloud customers crap.
lisper 16y ago
It's even worse than that: http://rondam.blogspot.com/2010/03/danger-will-robinson-rackspace-cloud.html http://rondam.blogspot.com/2010/03/danger-will-robinson-rack...
lzw 16y ago
The article makes the assumption that AWS will be reliable "because amazon runs on AWS". This has also been put forward by Amazon as a fact in their marketing materials. However, it simply isn't true. I worked at Amazon in the past, and knew many key people in the AWS team, as well as had my hands in the code bases behind Amazon.com, both generations. I only wish Amazon.com ran on an AWS infrastructure. This marketing claim is only true to the point that AWS operates out of the same datacenters that amazon.com ran.... but the code behind Amazon.com is/was much too old to run on AWS, and really not designed to operate on cloud services like AWS. I'm sure in the intervening years (I left several years ago) more and more projects have migrated to AWS. But at the time I worked there, substantially none of the systems behind Amazon.com were on AWS services. I not only wrote some of the code in them, but I also got way too deep into the deployment mechanics of Amazon.com. I can say this as a fact. Unfortunately, this dishonest marketing claim-- upon which many startups have decided that AWS is proven-- is about typical for the lack of integrity that Amazon as a business shows. They have great customer service, but you really don't want to know how the sausage is made. That said, this is probably a good thing, as AWS is a modern code base (by amazon standards at least) and run by a group of people who I consider to be better than the average engineers at Amazon. I can say this publicly because the truth is a positive defense, and I'm not revealing anything that couldn't be discovered with a bit of detective work. PS- I'm a small time customer of both services and I won't comment on their relative quality. I think both have issues and these issues are better commented on by people who have more extensive experience. I'm just dismayed that people believe AWS was "proven" by being "the same code that runs amazon.com", which is a complete and utter falsehood perpetrated by Amazon.
xal 16y ago
My honest question is why there is this odd loyalty to virtual environments in this community. I realize that it may be boring but you guys are passing up insane savings that can be had by using colocation. All cloud providers are very expensive when you actually do the math and you need more then 10 servers. Our example may be a bit extreme, but we are just building out a new datacenter at a colocation and will recover the entire up front investment ( about 150k, we have the cash to not need leasing ) in a bit over half a year.
- chopsueyar 16y ago
  Can you provide some details on your infrastructure?
- alttab 16y ago
  We also use colocation. I'm not a developer on the team that uses the infrastructure but we use OnRamp in Austin, TX (http://www.onr.com/ http://www.onr.com/) and I have rarely heard of any issues. Most delays are from international CDN requests which I believe we have also outsourced to speed it up.
- spidaman 16y ago
  There's a big difference between the cost to install and operate and the cost to scale. If your hardware utilization characteristics are easy to calibrate for and capacity requirements are predictably steady, you can certainly optimize your own racks at a colo. However, hardware changes quickly, businesses typically change even quicker and capacity requirements are often highly variable. Unless the above conditions are met, it's insanely risky to go into a contract for a colo. Let us know how that one year investment recovery works out for you.
- tdavis 16y ago
  Dedicated (rented) offers similar performance improvements and cost reductions without the extreme upfront cost (obviously costing more over its lifetime as a result). But these days it really seems like a blind spot. Maybe it's just that I never completely jumped on the cloud bandwagon (or was running servers far before it got rolling), but I really haven't found a generic use case for cloud hosting. Planning infrastructure upgrades isn't rocket science; you'll have more than 10 minutes' notice to get a new machine. There are certainly many specific use cases for cloud hosting, but as a generic hosting tier I find it incredibly overused—and companies waste money and are forced to deal with unreliable performance (EBS i/o anyone?!) as a result.
zemaj 16y ago
I moved all my services from EC2 to Rackspace Cloud about 2 years ago, but I'm regretting it. Rackspace Cloud does one thing well - small instances have great value CPU & local IO performance. If your app is CPU or local IO bound, splitting it across multiple 256MB instances on Rackspace Cloud will get you huge performance relative to price. I've been worried that this would degrade as the service grew, but that hasn't been the case. Unfortunately many other 'features' of Rackspace Cloud have been poor to awful recently. Some anecdotal stories; 1) We haven't been able to make images of our server or restore ANY backup of our servers for months. There is a bug in the backup service where if you have spaces in the names of your Cloud Files containers (completely unrelated to the backup service) then all images fail to be able to be restored. We can't remove the spaces in the containers because you can't rename containers (only delete) and there's too much data tied to different parts of our infrastructure in there. 2) In relation to the issue above, we have had a ticket open for over 2 months which we continually post updates with new information & asking for issue resolution. We never receive updates to the ticket itself and only receive information when contacting their live chat. The response is always "we're working on it". I could live with it if this was a short period, or not an absolutely vital part of their service, but come on - all backups broken for 2 months! No timelines on resolution. No ticket responses. No happy. 3) While CPU value is great on small instances you get the other end of the stick on large instances as other posters here have said. You don't get significantly improved performance above the 2GB servers. CPU capacity certainly does not double as their documents say. 4) Cloud Files latency is awful. Individual read/writes take 300-1000ms. Fine for a small number of large files. Impossible for a large number of small files. (Having said that, being able to upload files and publish to CDN in a click has saved me lots of time for static files I need to quickly publish). 5) Resizing mid to large instances is impossible. We recently tried to resize a 1GB (40GB disc) server to a 2GB (80GB disc) and it took OVER 3 DAYS. No really. It didn't complete. The resize process takes the server down towards the end. We had to get Rackspace to cancel the resize and manually spin up another server and transfer the files ourselves. To make it worse, we couldn't act on this issue initially because Rackspace insisted that the process was "almost complete" from 12 hours onwards. 2.5 days later we just gave up. We managed to do the manual transfer ourselves in a couple of hours. Even worse Rackspace seemed to not think that it was unusual for the process to take 3 days or express any desire to investigate further. 6) The web interface has awful performance at scale. Once you go above 20 cloud servers every single page load takes 10+ seconds. As the original poster says, the number of errors it spits out about not being able to complete operations is insane. It's rare I can go in there planning on doing something and not have to contact support to fix something broken on their end. 7) They're taking the entire web interface and API offline for 12 hours this week! You won't be able to spin up or take down any of your servers. Why? So they can fix a billing issue related to Cloud Sites (a service we don't use). I've always been a champion of Rackspace Cloud and Rackspace in general, but sadly I would no longer recommend them to people. I'm starting to make contingency plans and looking for other providers again.
jread 16y ago
Another distinguishing factor between Rackspace Cloud and EC2 or Linode are bandwidth caps. Rackspace limits outbound public network throughput to 10mbps for 256mb instances to 70mbps for 16gb instances. EC2 and Linode both provide an uncapped GigE Internet uplink for instances of any size. http://cloudservers.rackspacecloud.com/index.php/Frequently_Asked_Questions#Is_there_a_throughput_limit_on_my_server.27s_network_interface_card.3F http://cloudservers.rackspacecloud.com/index.php/Frequently_...
- megaman821 16y ago
  This actually becomes really important when you set up a small server as a proxy sever. We had to bump up our HAProxy server to a 512mb instance in order to get adequate bandwidth to serve our sites.
- jfine 16y ago
  They both limit internal network speeds also. EC2's limits are much higher than RS's.
mikey_p 16y ago
I'd also add that their billing software can't keep things straight. I've had a couple of servers that I spun up for a demo at a meetup, that somehow ended up with the same name, this prevented me from deleting them, and tells me that their control panel has concurrency issues, since you shouldn't be able to create two servers with the same name. I didn't notice the issue until 2 months later (I thought I had successfully deleted the servers) because all of the sudden I received huge bill that contained over 1100 hours of usage for each instance, for that month alone. WTF? Turns out their software failed to bill me the previous month so I didn't notice any change. Their response to my ticket about not being able to delete servers was to tell me the steps that I had to take to fix it (renaming the servers). I really wish when you had a ticket for stuff like that they'd actually act on it instead of just telling you how to fix stuff and expecting you to do it yourself.

203 points by suhail 16y ago

powdahound 16y ago

Goodbye code.mixpanel.com too? Page won't load. :(

suhail 16y ago
funny part is...it's on the rackspace cloud. Though probably our fault since we gave it a crappy box. Edit: 512MB of RAM should be enough...
- riffer 16y ago
  What is the limiting factor here? I am curious because it seems like almost any configuration & setup should be able to serve a blog post to the volume of request that come with being at the top of HN.
- suhail 16y ago
  Apache2 basically just OOM'd
  - madaxe 16y ago
    You misconfigured it, then, I'm afraid to say... Number of threads = server memory/maximum memory usage per thread - 10% margin for overheads.
  - chaosmachine 16y ago
    Disable keepalives: http://www.kalzumeus.com/2010/06/19/running-apache-on-a-memory-constrained-vps/ http://www.kalzumeus.com/2010/06/19/running-apache-on-a-memo...
redorb 16y ago
google-cache-version http://webcache.googleusercontent.com/search?sourceid=chrome&ie=UTF-8&q=cache:code.mixpanel.com/amazon-vs-rackspace/ http://webcache.googleusercontent.com/search?sourceid=chrome...
seldo 16y ago
Yeah, not exactly a ringing endorsement of rackspace that the rackspace-hosted blog saying they're leaving is down under the load.

barrydahlberg 16y ago

I'm suprised to see a setup using 50+ instances running on RackSpace cloud, wouldn't it have made sense to start moving towards dedicated servers by then?

cperciva 16y ago
Not necessarily. Sometimes flexibility is worth more than having the minimum possible cost per clock cycle.
- barrydahlberg 16y ago
  True but in theory the advantage with RackSpace is you can mix and match to balance the cost of your known requirements with the flexiblity for your unknown.
suhail 16y ago
We need the elasticity. I can't wait 24 hours for new boxes to come.
- tnm 16y ago
  Capacity planning. It's a thing. Try it out.
  - deleted 16y ago
    [deleted]
  - suhail 16y ago
    Capacity planning is hard when data volume doubles / triples over night.
    - jamesgolick 16y ago
      The fundamentals of capacity planning do not change based on the magnitude of your data growth. Why would they? We're mostly talking about looking at your data growth curve and extrapolating points in the future. Why would that become impossible just because the curve is steep?

powdahound 16y ago

Make sure you explore the limits of EBS before assuming it's a perfect solution. We've found it to have incredibly slow throughput at times. Some reference links: - http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs - http://victortrac.com/EC2_Ephemeral_Disks_vs_EBS_Volumes - http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html

gyardley 16y ago
We had these issues as well. I'd certainly try AWS, but I'd wait a couple of months before shelling out the upfront cost for the Reserved Instances.
- blantonl 16y ago
  EBS definitely has performance limitations, however creating striped RAID arrays of EBS volumes significantly helps with performance. This blog post walks through a bunch of things you can do to make the best out of the flexibility you get with EBS, and includes scheduler tuning, RAID block size planning, etc. http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs/ http://orion.heroku.com/past/2009/7/29/io_performance_on_ebs... My team runs a large AWS/EBS backed implementation and, while we've seen some of the EBS performance issues, we've certainly engineered past them to get us where we need to be.
  - chrisbolt 16y ago
    Wouldn't creating striped RAID arrays of EBS volumes add to the cost and remove the ability to do snapshots? How do you get a consistent snapshot of multiple striped EBS volumes?
    - spudlyo 16y ago
      It just makes it more complicated. You have lock and snap all of the volumes at once. The ec2-consistent-snapshot tool makes this easier to manage.
    - 16y ago

cperciva 16y ago

Quoth the article: Lastly, we moved over to the Rackspace Cloud because they cut a deal with YCombinator (one of the many benefits of being part of YC). Can anyone say what this deal is, or is it secret?

varenc 16y ago
Just reduced price hosting. My guess is its a deal that won't be happening again? :-)
mikeyur 16y ago
For a separate project I've worked on (non-YC), we negotiated 50% off bandwidth and somewhat reduced server rates. You just need to ask for it. Not sure what the YC deal is, but I assume it's similar.
- aaroneous 16y ago
  I had a similar experience: with minimal negotiating we were able to get 50% off their initial quote. I also renegotiated our terms later within the contract period with relative ease. They pad their pricing pretty heavily, so don't be afraid to ask.
mattlong 16y ago
Can't speak for other YC companies (it may vary from batch to batch), but for us it was a generous rackspace cloud monthly credit for 12 months.

joecode 16y ago

One potential reason not to move: Security. A good friend deep in the security community once told me, off hand, that EC2 was "owned." I didn't take this too seriously until another good friend, who has been working at Amazon for the past several years, told me that engineers at Amazon were generally forbidden from using AWS due to security concerns. That much said, I still decided to use EC2/RDS/S3 to host the infrastructure of my latest startup. It is just too convenient to walk away from. Once it matters, I can move the critical stuff to dedicated servers. EDIT: To clarify, I'm not suggesting that Amazon knows AWS is "owned" and offers it to others anyway. I'm only noting that, for certain critical services, they themselves do not appear willing to take the risk.

rbranson 16y ago
While this is disconcerting, I wouldn't make any business decisions based on such a claim. The idea that EC2 is "owned" without Amazon knowing about it is closing in on absurdity. I've worked directly with Amazon as an outside vendor and they are very security concious, to the point of near paranoia.
- joecode 16y ago
  While I agree it is hard to believe, it would be even more surprising if Amazon did know about it. The fact that they do not use AWS internally suggests that---at least with their level of paranoia---they seem to suspect AWS themselves.
  - benblack 16y ago
    Both you and your friend are misinformed.
    - joecode 16y ago
      I'm not sure how you can say that so matter-of-factly. My security friend was talking about something that Amazon does not (and presumably very few people do) know. Meanwhile my friend at Amazon was just stating the fact that he was not supposed to use AWS, or only with extreme caution. Of course that may differ from department to department, if that's what you mean.

yesimahuman 16y ago

Another control panel complaint: DNS. For some reason beyond me you have to choose a server just to configure DNS. Added on top of the fact that the control panel is really slow, it just becomes a pain to use.

nl 16y ago
Yes!! I was trying to do a DNS change and I couldn't find the DNS panel anywhere until I Google'd it.

jread 16y ago

CPU is a major bottleneck for Rackspace Cloud. All instance sizes get the same 4 cores and about the same compute resources. CPU performance is roughly the same on a 1GB cloud server as an 8GB cloud server, you are just paying for more memory. Rackspace also uses ONLY Opteron 2374 2.20 GHz processors. EC2 on the other hand offers linear CPU performance improvement on larger sized instances. EC2 also uses a heterogenous hardware ranging from Opteron 2218 or Xeon E5430 for m1 instances; Xeon E5410 for c1 instances; Xeon X5550 for m2 instances and Xeon X5570 (hyper-threaded to 16 cores) for the cluster compute instances. EBS on the cluster instance is also much faster than local disk IO in the Rackspace Cloud based on testing I've done (due to non-blocking 10G network). Here are a couple of references for this: http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchmarking-in-cloud.html http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma... http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in-cloud.html http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in...

jread 16y ago

<quote>Amazon has a CDN and servers distributed globally. This is important to Mixpanel as websites all over the world are sending us data. There’s nothing like this on Rackspace.</quote> Actually, Rackspace cloud offers CDN services with Cloud Files through Limelight, although it does not support some features that CloudFront does like CName and streaming.

suhail 16y ago
But not physical servers which is really what I am referring to.
- jread 16y ago
  Rackspace Cloud has data centers in Texas and Illinois. However, they don't let you choose (your account is assigned to one data center at the time of creation). EC2 certain offers a lot more flexibility in this regard with EC2 data center regions in California, Virginia, Ireland and Singapore. I spoke with Rackspace at Interop a few weeks ago and they told me they are working on expanding to additional data centers including an international data center very soon and offering choice.
  - 0x44 16y ago
    You can "choose" insofar as you can email support and they can manually change what data-center your next VM and all subsequent ones will be built in. It really should be exposed through the API, but the people who write the customer-facing API haven't done it.
    - jread 16y ago
      That is not the story I got when I contacted support and asked if I could deploy a VM specifically to the Chicago cloud. Rackspace's support told me this was not possible an I'd have to setup an entirely new account in order to deploy to that data center.
      - 16y ago

adriand 16y ago

This is a rather unpleasant review of a service we were about to move over to. For companies that have moderate performance requirements (e.g. visitors in the range of 30k or 40k per day across a range of web apps and sites), reasonable but by no means expert level server administration skills, the need for a redundant environment to satisfy SLAs with clients (e.g. two app servers + load balancer + master/slave db servers), and the desire to focus mainly on software development instead of server admin, what companies does the HN community recommend? We've been considering Rackspace Cloud and Linode, but are open to any suggestions. We also have a quote for a standard, managed four-server + hardware load balancer deployment in front of us but it is pricey ($3000+/month).

sanswork 16y ago
I've used Rackspace, Amazon and GoGrid. I'm currently almost entirely on GoGrid and just finishing up migrating back off of Amazon for the second time(Migrated some services back to them when they opened west coast). There have been some stability issues in the past year(instances going down for maintenance regularly) but haven't had a problem with that in a while and was advised it was due to them upgrading all the servers in their cluster. The ability to mix dedicated and cloud systems easily has been the killer feature for me. We also prepay and the number of servers we get for the cost is pretty good.
madaxe 16y ago
Linode are wonderful - best support I've had from anyone, ever. Nice instances, too!
- shazow 16y ago
  +1 to Linode.
- catch23 16y ago
  another +1. we moved to linode from rackspace. They don't have that crazy backup limitation that rackspace does, and they're always open to custom inquiries. Rackspace has the big company mentality -- if our interface doesn't have an option for it, it doesn't exist.

newobj 16y ago

Wow, that's a brutal under-the-bus-tossing. But well deserved. Most of the points are spot-on. I've operated a several dozen machine fleet for 2+ years on EC2 and I can tell you that the number of times boxes have gone down never to come back (which most people tend to think happens regularly on EC2) is incredibly small. We actually run a Hadoop cluster (including DFS) on a spot instances. We never lose the spot bid. We pay way less than the going rate for the compute time. Less than the reserved instance rate. It's awesome. Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye. It's not too hard. I would second another commenter's caution on EBS though. We never put it into production. Personally I never experienced an ephemeral drive failure that had repercussions - when they (rarely) occurred, the drive was RAIDed or in our Hadoop cluster (e.g. redundant). We made two experiments into using EBS with our DB's, and both times, literally within 24 hours, we experienced a catastrophic failure of the EBS volume, one time unrecoverable. So that put me pretty well off EBS. I can't say that the performance is necessarily the best, and we do experience the occasional odd asymmetrical inter-machine latencies (e.g. 300ms to establish a tcp connection in one direction, but normal <1ms in the other), but for the most part AWS is just awesome.

gruseom 16y ago
Yes, you obviously need a plan to deal with your cluster vanishing in the blink of an eye. How long do the spot instances typically run before they vanish?
- cperciva 16y ago
  A spot instance should stay around until the auction price is higher than what you bid. That price has been quite stable recently, so as long as you bid slightly higher than that rate you'll probably have your instance for months.
  - gruseom 16y ago
    That's better than I was expecting. I'm surprised more people don't take advantage of this (or maybe they do, I don't know). Use S3 for permanent storage, ephemeral disk and RAM for cache, and pay spot rates. Sounds pretty good if one's design allows for it.

dangrover 16y ago

Rackspace cloud's DNS stuff stinks. No way to add TXT records -- you have to open a ticket! Sure you can host it yourself, but every other cloud provider has this in their UI. I get the feeling they're just in "maintenence mode" over there and don't have anyone working hard on improving the offerings.

danudey 16y ago
Their DNS used to be great, if simple. You could put in a record for your new server and by the time you'd logged in, the record was live. Now, it's gotten to the point where their DNS updates so slowly that I just re-use old subdomains instead of creating new ones, just so that I can get my project tested today rather than tomorrow. I'm working on moving everything over to Amazon; more flexibility. I like Rackspace Cloud, but they're falling behind and they either don't know, don't care, or can't catch up.
megaman821 16y ago
What is even more sad, is that their DNS console for hosted servers and Slicehost is so much better. They are literally sitting on something better and giving their cloud customers crap.

lisper 16y ago

It's even worse than that: http://rondam.blogspot.com/2010/03/danger-will-robinson-rackspace-cloud.html http://rondam.blogspot.com/2010/03/danger-will-robinson-rack...

lzw 16y ago

The article makes the assumption that AWS will be reliable "because amazon runs on AWS". This has also been put forward by Amazon as a fact in their marketing materials. However, it simply isn't true. I worked at Amazon in the past, and knew many key people in the AWS team, as well as had my hands in the code bases behind Amazon.com, both generations. I only wish Amazon.com ran on an AWS infrastructure. This marketing claim is only true to the point that AWS operates out of the same datacenters that amazon.com ran.... but the code behind Amazon.com is/was much too old to run on AWS, and really not designed to operate on cloud services like AWS. I'm sure in the intervening years (I left several years ago) more and more projects have migrated to AWS. But at the time I worked there, substantially none of the systems behind Amazon.com were on AWS services. I not only wrote some of the code in them, but I also got way too deep into the deployment mechanics of Amazon.com. I can say this as a fact. Unfortunately, this dishonest marketing claim-- upon which many startups have decided that AWS is proven-- is about typical for the lack of integrity that Amazon as a business shows. They have great customer service, but you really don't want to know how the sausage is made. That said, this is probably a good thing, as AWS is a modern code base (by amazon standards at least) and run by a group of people who I consider to be better than the average engineers at Amazon. I can say this publicly because the truth is a positive defense, and I'm not revealing anything that couldn't be discovered with a bit of detective work. PS- I'm a small time customer of both services and I won't comment on their relative quality. I think both have issues and these issues are better commented on by people who have more extensive experience. I'm just dismayed that people believe AWS was "proven" by being "the same code that runs amazon.com", which is a complete and utter falsehood perpetrated by Amazon.

xal 16y ago

My honest question is why there is this odd loyalty to virtual environments in this community. I realize that it may be boring but you guys are passing up insane savings that can be had by using colocation. All cloud providers are very expensive when you actually do the math and you need more then 10 servers. Our example may be a bit extreme, but we are just building out a new datacenter at a colocation and will recover the entire up front investment ( about 150k, we have the cash to not need leasing ) in a bit over half a year.

chopsueyar 16y ago
Can you provide some details on your infrastructure?
alttab 16y ago
We also use colocation. I'm not a developer on the team that uses the infrastructure but we use OnRamp in Austin, TX (http://www.onr.com/ http://www.onr.com/) and I have rarely heard of any issues. Most delays are from international CDN requests which I believe we have also outsourced to speed it up.
spidaman 16y ago
There's a big difference between the cost to install and operate and the cost to scale. If your hardware utilization characteristics are easy to calibrate for and capacity requirements are predictably steady, you can certainly optimize your own racks at a colo. However, hardware changes quickly, businesses typically change even quicker and capacity requirements are often highly variable. Unless the above conditions are met, it's insanely risky to go into a contract for a colo. Let us know how that one year investment recovery works out for you.
tdavis 16y ago
Dedicated (rented) offers similar performance improvements and cost reductions without the extreme upfront cost (obviously costing more over its lifetime as a result). But these days it really seems like a blind spot. Maybe it's just that I never completely jumped on the cloud bandwagon (or was running servers far before it got rolling), but I really haven't found a generic use case for cloud hosting. Planning infrastructure upgrades isn't rocket science; you'll have more than 10 minutes' notice to get a new machine. There are certainly many specific use cases for cloud hosting, but as a generic hosting tier I find it incredibly overused—and companies waste money and are forced to deal with unreliable performance (EBS i/o anyone?!) as a result.

zemaj 16y ago

I moved all my services from EC2 to Rackspace Cloud about 2 years ago, but I'm regretting it. Rackspace Cloud does one thing well - small instances have great value CPU & local IO performance. If your app is CPU or local IO bound, splitting it across multiple 256MB instances on Rackspace Cloud will get you huge performance relative to price. I've been worried that this would degrade as the service grew, but that hasn't been the case. Unfortunately many other 'features' of Rackspace Cloud have been poor to awful recently. Some anecdotal stories; 1) We haven't been able to make images of our server or restore ANY backup of our servers for months. There is a bug in the backup service where if you have spaces in the names of your Cloud Files containers (completely unrelated to the backup service) then all images fail to be able to be restored. We can't remove the spaces in the containers because you can't rename containers (only delete) and there's too much data tied to different parts of our infrastructure in there. 2) In relation to the issue above, we have had a ticket open for over 2 months which we continually post updates with new information & asking for issue resolution. We never receive updates to the ticket itself and only receive information when contacting their live chat. The response is always "we're working on it". I could live with it if this was a short period, or not an absolutely vital part of their service, but come on - all backups broken for 2 months! No timelines on resolution. No ticket responses. No happy. 3) While CPU value is great on small instances you get the other end of the stick on large instances as other posters here have said. You don't get significantly improved performance above the 2GB servers. CPU capacity certainly does not double as their documents say. 4) Cloud Files latency is awful. Individual read/writes take 300-1000ms. Fine for a small number of large files. Impossible for a large number of small files. (Having said that, being able to upload files and publish to CDN in a click has saved me lots of time for static files I need to quickly publish). 5) Resizing mid to large instances is impossible. We recently tried to resize a 1GB (40GB disc) server to a 2GB (80GB disc) and it took OVER 3 DAYS. No really. It didn't complete. The resize process takes the server down towards the end. We had to get Rackspace to cancel the resize and manually spin up another server and transfer the files ourselves. To make it worse, we couldn't act on this issue initially because Rackspace insisted that the process was "almost complete" from 12 hours onwards. 2.5 days later we just gave up. We managed to do the manual transfer ourselves in a couple of hours. Even worse Rackspace seemed to not think that it was unusual for the process to take 3 days or express any desire to investigate further. 6) The web interface has awful performance at scale. Once you go above 20 cloud servers every single page load takes 10+ seconds. As the original poster says, the number of errors it spits out about not being able to complete operations is insane. It's rare I can go in there planning on doing something and not have to contact support to fix something broken on their end. 7) They're taking the entire web interface and API offline for 12 hours this week! You won't be able to spin up or take down any of your servers. Why? So they can fix a billing issue related to Cloud Sites (a service we don't use). I've always been a champion of Rackspace Cloud and Rackspace in general, but sadly I would no longer recommend them to people. I'm starting to make contingency plans and looking for other providers again.

jread 16y ago

Another distinguishing factor between Rackspace Cloud and EC2 or Linode are bandwidth caps. Rackspace limits outbound public network throughput to 10mbps for 256mb instances to 70mbps for 16gb instances. EC2 and Linode both provide an uncapped GigE Internet uplink for instances of any size. http://cloudservers.rackspacecloud.com/index.php/Frequently_Asked_Questions#Is_there_a_throughput_limit_on_my_server.27s_network_interface_card.3F http://cloudservers.rackspacecloud.com/index.php/Frequently_...

megaman821 16y ago
This actually becomes really important when you set up a small server as a proxy sever. We had to bump up our HAProxy server to a 512mb instance in order to get adequate bandwidth to serve our sites.
jfine 16y ago
They both limit internal network speeds also. EC2's limits are much higher than RS's.

mikey_p 16y ago

I'd also add that their billing software can't keep things straight. I've had a couple of servers that I spun up for a demo at a meetup, that somehow ended up with the same name, this prevented me from deleting them, and tells me that their control panel has concurrency issues, since you shouldn't be able to create two servers with the same name. I didn't notice the issue until 2 months later (I thought I had successfully deleted the servers) because all of the sudden I received huge bill that contained over 1100 hours of usage for each instance, for that month alone. WTF? Turns out their software failed to bill me the previous month so I didn't notice any change. Their response to my ticket about not being able to delete servers was to tell me the steps that I had to take to fix it (renaming the servers). I really wish when you had a ticket for stuff like that they'd actually act on it instead of just telling you how to fix stuff and expecting you to do it yourself.