9 ms·

I'm sure Amazon will fix this issue, but the question it brings to my mind is- why are so many people using ECC? Most startups need hosting, not an elastic com

by nirvana 15y ago

I'm sure Amazon will fix this issue, but the question it brings to my mind is- why are so many people using ECC? Most startups need hosting, not an elastic compute cloud. ECC makes sense for someone who needs to spin up 1,000 workers, pull data out of S3, process it, store the results in S3 and then spin down the workers when it is done. But startups need hosts that are up 24/7. ECC doesn't give you any guarantee of uptime, and if it goes down the local (fast) disk is ephemeral. Yet, the EBS alternative which is backed up, is very slow. The basic VPS offerings, that Linode, Rackspace, and just about everyone else offers, isn't available at all from Amazon (near as I can see.) Yet this is what startups need- local disks, a small monthly fee, and up all the time. So, Amazon requires extra engineering--to account for nodes going down more often and to make ephemeral disks reliable or EBS performant. It also puts you on the path to lock-in, since so many of amazons services have their own unique APIs. It isn't exactly cheap, near as I can tell, when compared to, say, dedicated hosting in germany. Its advantages exist elsewhere- if you need to spin up a bunch of machines with an API you can do this at rackspace cloud. And... that's about the only advantage I'm able to think of. Personally, I'm architecting to have some overcapacity built in and to survive a spike, because I use that excess capacity in the off hours for heavy lifting. I wouldn't try to bring up extra nodes in the morning and shut them down in the evening anyway... and I doubt that many startups are really doing that. Possibly, I'm missing something. I tend to forget about features that aren't compelling to me, but are compelling to others. So maybe there's something that's important to these startups.

petervandijck 15y ago
S3 is a big part of it. If you're storing lots of data, a "local disk" just isn't enough. Now you're spending your time solving problems Amazon already solved. Occasional downtime just isn't that big of an issue for many startups, who are still looking for product-market fit.
- nirvana 15y ago
  I see what you're saying, and I failed to illuminate that I'm coming from the perspective of someone who is working with a cluster of riak nodes. Riak is a ring topology much inspired by the Amazon S3 design. I'd never trust anything to just one spinning rust device, sure. I was taking a distributed storage architecture (using whatever open source platform you prefer) for granted. But I recognize that S3 predates many of them.
tybris 15y ago
If you expect to grow your start-up quickly, it makes sense to use a provider that allows you to scale. It is very costly to maintain two separate infrastructures for different providers.
- nirvana 15y ago
  I think I'm misunderstanding your point. Why would you need two different infrastructures? Anything you can host on ECC you can host on backspace cloud, but without being ephemeral. Or you can order servers by the dozen from other providers. Maybe you're talking about ordering servers by the hundreds, but are successful startups like foursquare doing that? Seems a few a week is what they'd add, and you could do that with most other hosts, without the added complexity of Amazon.
  - tybris 15y ago
    The difficulty of scaling is not in getting the hardware, but in the operational burden and frequency of failure. With EBS, machine and disk failures are effectively irrelevant and you can instantly migrate to another machine. You don't need to worry much about data loss. If a disk breaks in one of your machines, EBS keeps working. If the machine breaks, the data is still in EBS. If EBS breaks, the back-up is still in S3. Dedicated hosting and rackspace cloud may be cheaper, but if you expect to be in a situation where failure becomes frequent, you'd better be on EC2.
    - nirvana 15y ago
      With AWS, the development, operational and financial burdens are higher, and the frequency of failure is much higher. EBS is not a unique feature, as there are multiple equivalents of EBS, along with higher level methods of replicating data, namely, couchDB, riak, cassandra, etc. But I think you've answered my question- people use AWS because they don't realize this.
      - tybris 15y ago
        If you are starting up, where are you going to get the capital, expertise, and people to operate and maintain those systems? You have better things to do.
snorkel 15y ago
Having used EC2 for web hosting I can affirm that you are exactly right. Web hosting on EC2 is a challenge and quite expensive. From my experience EC2 is much better suited for large data storage, distributed computing, and any other use case that can tolerate EC2 quirks, hiccups, and outages.
cookiecaper 15y ago
As tybris's comment indicates, the allure is primarily that you can configure the system to automatically spin up new instances on-the-fly as load increases, in case your startup gets on the front page of reddit. Because most startups want to gratify that vanity, they set up on EC2 because they want to prepare for the "inevitable" deluge of users without paying for an adequate pipe or server configuration around-the-clock.

by nirvana 15y ago

petervandijck 15y ago

S3 is a big part of it. If you're storing lots of data, a "local disk" just isn't enough. Now you're spending your time solving problems Amazon already solved. Occasional downtime just isn't that big of an issue for many startups, who are still looking for product-market fit.

nirvana 15y ago
I see what you're saying, and I failed to illuminate that I'm coming from the perspective of someone who is working with a cluster of riak nodes. Riak is a ring topology much inspired by the Amazon S3 design. I'd never trust anything to just one spinning rust device, sure. I was taking a distributed storage architecture (using whatever open source platform you prefer) for granted. But I recognize that S3 predates many of them.

tybris 15y ago

If you expect to grow your start-up quickly, it makes sense to use a provider that allows you to scale. It is very costly to maintain two separate infrastructures for different providers.

nirvana 15y ago
I think I'm misunderstanding your point. Why would you need two different infrastructures? Anything you can host on ECC you can host on backspace cloud, but without being ephemeral. Or you can order servers by the dozen from other providers. Maybe you're talking about ordering servers by the hundreds, but are successful startups like foursquare doing that? Seems a few a week is what they'd add, and you could do that with most other hosts, without the added complexity of Amazon.
- tybris 15y ago
  The difficulty of scaling is not in getting the hardware, but in the operational burden and frequency of failure. With EBS, machine and disk failures are effectively irrelevant and you can instantly migrate to another machine. You don't need to worry much about data loss. If a disk breaks in one of your machines, EBS keeps working. If the machine breaks, the data is still in EBS. If EBS breaks, the back-up is still in S3. Dedicated hosting and rackspace cloud may be cheaper, but if you expect to be in a situation where failure becomes frequent, you'd better be on EC2.
  - nirvana 15y ago
    With AWS, the development, operational and financial burdens are higher, and the frequency of failure is much higher. EBS is not a unique feature, as there are multiple equivalents of EBS, along with higher level methods of replicating data, namely, couchDB, riak, cassandra, etc. But I think you've answered my question- people use AWS because they don't realize this.
    - tybris 15y ago
      If you are starting up, where are you going to get the capital, expertise, and people to operate and maintain those systems? You have better things to do.

snorkel 15y ago

Having used EC2 for web hosting I can affirm that you are exactly right. Web hosting on EC2 is a challenge and quite expensive. From my experience EC2 is much better suited for large data storage, distributed computing, and any other use case that can tolerate EC2 quirks, hiccups, and outages.

cookiecaper 15y ago

As tybris's comment indicates, the allure is primarily that you can configure the system to automatically spin up new instances on-the-fly as load increases, in case your startup gets on the front page of reddit. Because most startups want to gratify that vanity, they set up on EC2 because they want to prepare for the "inevitable" deluge of users without paying for an adequate pipe or server configuration around-the-clock.