4 ms·

Wow $5000/mo for 2000rps, just for the application servers? That's absurd. I think we're paying around $2000/mo for our app servers, a database which is over 2

by iLoch 10y ago

Wow $5000/mo for 2000rps, just for the application servers? That's absurd. I think we're paying around $2000/mo for our app servers, a database which is over 2TB in size, and we ingest about 10 megabytes of text data per second, on top of a couple thousand requests per second to the user facing application. I don't think we'll be turning to Heroku any time soon. Sorry if this seems like a pissing contest, just giving some basis for comparison.

Rapzid 10y ago
Do you ingest that through rails? Maybe you do, IDK! But srsly, comparing different use cases as apples to apples is not very informative. I could give you our numbers at 300rps REAL peak and tell you we pay at least a 2x for our hosting.. But that tells you nothing about our latency percentiles, CND costs for MANY TB/month, burst capacity at reasonably latencies, etc. And that's not accounting for the fact that certain CRITICAL code paths have different bottlenecks depending on who hits them due to lock times. Different workloads, different companies, different requirements! Don't be so dismissive!
- true_religion 10y ago
  A lot of what you're saying is true, but cloud platform providers are always going to be more expensive than a bare metal server. Heroku is amongst the least commoditized and more expensive providers out there.
ukd1 10y ago
This sounds like classic engineer(s) talk; totally ignoring the human cost of setup, maint and ongoing development which is usually handled by an (expensive) ops team. It's not $5k vs $2k, it's $5k vs $2k + ops time.
- jacques_chester 10y ago
  If you run Cloud Foundry (CF), you can get the Heroku experience on your own setup. It works on vSphere, OpenStack, AWS, Azure and GCP is getting there. More platforms are coming. I work at Pivotal, we donate the majority of engineering to CF. We also dogfood it by running Pivotal Web Services (PWS)[0], which is a Heroku competitor. One major difference: PWS (and CF) doesn't have fixed instance sizes. You can specify, to the megabyte, how much RAM you want your app to have. Other resources are dished out proportionally. PWS has thousands of customers, thousands of VMs under management by BOSH, I'm not sure how many apps and services at this point. Last I checked, it's run by about 8-10 people in two shifts (SF and Beijing). [0] http://run.pivotal.io/ http://run.pivotal.io/
- geebee 10y ago
  I'm not sure what the impact of $5k a month is on a site that needs to scale to 125k requests per minute (I really do mean I'm not sure, I'm not saying this to dismiss the cost). Total costs and revenue might put this in perspective, there are a lot of factors here.
- iLoch 10y ago
  It's not $5k vs. $2k + ops time. It's $5k + ops time vs. $2k + ops time. The notion that you don't have to do any maintenance work on a cloud setup is silly, and so too is the notion that a dedicated setup requires full time maintenance.
claudiug 10y ago
are you using rails stack. If so, can you explain how you achieve it?
- iLoch 10y ago
  .NET (99.99%) and some Go.
Gepsens 10y ago
A one master mesos cluster with 5 c4.2xlarge public nodes only costs 2k$...
- hhandoko 10y ago
  I have been considering Mesos for a side project (we're close to getting an amount of traffic that make sense for it). However, last I looked, persistent storage is still under active development and moving to a distributed database is not in our priority at the moment.
briandear 10y ago
Who much do you pay the guy to manage those servers?
- iLoch 10y ago
  There's very little management to be done. We're on Windows which is pretty hands off. But to answer your question, we don't have (or need) anyone dedicated to it.
technion 10y ago
There's some new serverless trend here that I must just be too old to understand. I was talking to a group in a startup last week, who had ~40 developers, and they looked horrified when I asked why, at their size, they were still running Heroku. They went on to equate "running your own VPS" with "writing your own crypto code", as though it was a cardinal sin for mere mortals.

by iLoch 10y ago

Rapzid 10y ago

Do you ingest that through rails? Maybe you do, IDK! But srsly, comparing different use cases as apples to apples is not very informative. I could give you our numbers at 300rps REAL peak and tell you we pay at least a 2x for our hosting.. But that tells you nothing about our latency percentiles, CND costs for MANY TB/month, burst capacity at reasonably latencies, etc. And that's not accounting for the fact that certain CRITICAL code paths have different bottlenecks depending on who hits them due to lock times. Different workloads, different companies, different requirements! Don't be so dismissive!

true_religion 10y ago
A lot of what you're saying is true, but cloud platform providers are always going to be more expensive than a bare metal server. Heroku is amongst the least commoditized and more expensive providers out there.

ukd1 10y ago

This sounds like classic engineer(s) talk; totally ignoring the human cost of setup, maint and ongoing development which is usually handled by an (expensive) ops team. It's not $5k vs $2k, it's $5k vs $2k + ops time.

jacques_chester 10y ago
If you run Cloud Foundry (CF), you can get the Heroku experience on your own setup. It works on vSphere, OpenStack, AWS, Azure and GCP is getting there. More platforms are coming. I work at Pivotal, we donate the majority of engineering to CF. We also dogfood it by running Pivotal Web Services (PWS)[0], which is a Heroku competitor. One major difference: PWS (and CF) doesn't have fixed instance sizes. You can specify, to the megabyte, how much RAM you want your app to have. Other resources are dished out proportionally. PWS has thousands of customers, thousands of VMs under management by BOSH, I'm not sure how many apps and services at this point. Last I checked, it's run by about 8-10 people in two shifts (SF and Beijing). [0] http://run.pivotal.io/ http://run.pivotal.io/
geebee 10y ago
I'm not sure what the impact of $5k a month is on a site that needs to scale to 125k requests per minute (I really do mean I'm not sure, I'm not saying this to dismiss the cost). Total costs and revenue might put this in perspective, there are a lot of factors here.
iLoch 10y ago
It's not $5k vs. $2k + ops time. It's $5k + ops time vs. $2k + ops time. The notion that you don't have to do any maintenance work on a cloud setup is silly, and so too is the notion that a dedicated setup requires full time maintenance.

claudiug 10y ago

are you using rails stack. If so, can you explain how you achieve it?

iLoch 10y ago
.NET (99.99%) and some Go.

Gepsens 10y ago

A one master mesos cluster with 5 c4.2xlarge public nodes only costs 2k$...

hhandoko 10y ago
I have been considering Mesos for a side project (we're close to getting an amount of traffic that make sense for it). However, last I looked, persistent storage is still under active development and moving to a distributed database is not in our priority at the moment.

briandear 10y ago

Who much do you pay the guy to manage those servers?

iLoch 10y ago
There's very little management to be done. We're on Windows which is pretty hands off. But to answer your question, we don't have (or need) anyone dedicated to it.

technion 10y ago

There's some new serverless trend here that I must just be too old to understand. I was talking to a group in a startup last week, who had ~40 developers, and they looked horrified when I asked why, at their size, they were still running Heroku. They went on to equate "running your own VPS" with "writing your own crypto code", as though it was a cardinal sin for mere mortals.