7 ms·

Why Stack Exchange Isn’t in the Cloud

126 points by johns 15y ago

coob 15y ago
So for cultural reasons? Is it not perhaps because there's much less competition for .NET stack cloud platforms, therefore less quality?
- droz 15y ago
  competition isn't required for quality.
  - bprieto 15y ago
    Of course it is. If there is no competition, you can get by with a crappy product or service if your client need it.
    - philwelch 15y ago
      You're arguing that competition is sufficient for quality, not that it's necessary. The first vendor might choose to provide a high quality product regardless, which could, in itself, provide a barrier to entry preventing competition. The real win of competition is that it allows a market to be commoditized, at which point it actually operates like we all learned in our Econ 101 class.
    - hvs 15y ago
      I wouldn't say that it is required, simply that it is more likely to improve quality.
    - Rexxar 15y ago
      I think droz mean "You don't need competition to do a good job" rather than "It's better to have competition when you want buy something of quality". But in either case, sometimes, competition lowers at the same time the price and the quality. So competition doesn't necessarily increase quality.
- hughesdan 15y ago
  Amazon has very good .NET support. So does Rackspace. So does Azure (obviously). In my personal experience the quality of cloud platforms has been comparable for both Linux and Windows environments. Is there any evidence that .NET cloud platforms are of lower quality? I'm genuinely interested. Please elaborate.
  - reuser 15y ago
    Personally, I doubt there is a real quality problem with those platforms. But running .NET on those platforms looks unnecessarily expensive to me. If I were married to Microsoft (as SO is) then I would avoid 'the cloud' for that reason. Given that one wishes to use Windows/.NET, I feel the decision follows pretty naturally from an analysis of costs - no vague argument about culture necessary
    - hughesdan 15y ago
      I would say cost (referring to license fees) is more relevant in deciding whether or not to choose the .NET stack in the first place. You're not going to avoid license fees by avoiding the cloud. Their culture argument is a funny one. But I have no beef with it. If owning and tinkering with the hardware makes them happy, more power to them. As an aside, I've been using Azure lately for my .NET projects. MSFT has done a nice job with that platform and don't get enough credit for it IMHO. I especially like SQL Azure.
MichaelGG 15y ago
What a silly argument. "If you just want to use someone else’s computers, it means you don’t love computers — at least not every aspect to them." Uh, don't they use Dell? If they love them so much, why not build their own? Why stop there? I love software and programming languages, but it doesn't mean I'm going to use my own compiler or run a company on my own libc.
- xradionut 15y ago
  Whom is going to do a better job? A group of educated, passionate people that can troubleshoot their own problems in realtime? Or "Peggy" at Cloud Vendor X that deals with 400 other clients?
  - mechanical_fish 15y ago
    That depends on whether the fix that Peggy deploys for one client also fixes the same problem for the other 399. In other words: It depends. There's no general answer, because deciding where to draw the abstraction barrier between you and your vendors is an unsolved, difficult, and evolving problem.
- KyleBrandt 15y ago
  There is always the question of degree and practicality. So in terms of libc / and custom compilers that is perhaps ad absurdum. There really wouldn't be any practical benefit taking it to that far for most companies. However, people that love programming languages often probably wish it was practical and that they had an excuse to write compilers at some point. As far as Dell goes, I honestly have mixed feelings at this point. When I started at the company there was less than 10 people, so the idea of having centralized firmware updates etc seemed practical. I'm not sure I really feel this way any more -- I go back forth. But sticking with Dell seems to make sense at this point so we have more uniformity in hardware and management.
  - josephcooney 15y ago
    A friend of mine has the uncanny ability of re-formulating every problem he is given in such a way that the logical solution to the problem is always: "Lets write a parser". He loves writing them. Not quite 'writing your own compiler' but it's going that way.
jtbigwoo 15y ago
I suspect that this is part of the problem with many big manufacturing companies these days. Ford was a leader for so many years was because their leadership was passionate about engineering and manufacturing. Henry Ford was an engineer for the Edison Company before he was a manager and is famous for obsessing about every detail of the manufacturing process. That sort of spirit infected the company up through the 1960's and 70's when the MBA's took over and everything was reduced to a column of numbers. It appears that many early tech companies (TI, HP) have headed in the same direction. Maybe 30 - 40 years is the limit before the accountants take over.
- mechanical_fish 15y ago
  A Ford bought today is less expensive, considerably safer, more efficient, and more reliable than a Ford from the days of Henry Ford. Meanwhile, today's manufacturing companies routinely make objects – like the processor driving the machine that you are reading this on – that are orders of magnitude more complex than the most advanced technology of 1959. And all of this was done by companies that employ lots of accountants. I'm not sure why you think that accountants and engineers don't routinely coexist, or that Henry Ford didn't employ plenty of accountants back in his day. Being able to manufacture good stuff in bulk at reasonable prices is a major exercise in accounting, and always has been.
  - Donwangugi 15y ago
    The current CEO of Ford, Alan Mulally is an engineer by trade. I attribute a lot of their current success to him.
jinushaun 15y ago
Poorly written argument full of red herrings. From what I can tell, their answer to the "why aren't you in the cloud" question is that they "love computers so much that they want to do it all themselves because it's so fun".
- mechanical_fish 15y ago
  Hey, at least they're being honest. Would you have preferred a gigantic stack of technical chaff designed to rationalize and obscure the fact that they're doing whatever seemed like fun at the time? Because I'm sure we can find you some examples. ;)
- nephyo 15y ago
  If I asked you why you were eating pizza and you responded "because I like pizza" would you still consider that a red herring argument? All they are saying is that they are not in the cloud because they don't want to be in the cloud and because they quite like not being in the cloud. It works for them. Why is that expression at all controversial? If they'd labeled their blog "Why being in the Cloud is a bad idea" or "Why being in the cloud would cost Stack Exchange a lot more" or "Why nobody should ever use the Cloud" then I could see the reason for the dispute. Then their arguments would be very unconvincing. It is certainly valid for Stack Exchange to make a choice because it fits within their culture and then explain that that is why they are doing it. You might think it's a dumb decision and that's fine. But they clearly are not trying to convince you that they've made the best possible decision. They're just saying that good or bad, they've made the decision that they want to make. And it's clearly a decision that has been working well for them so far anyway. So why do people keep bugging them about it?
  - jinushaun 15y ago
    You're correct about alternative titles for the blog post. However, the assumption is that on a technical blog, readers would expect technical answers--at least I did. You're not going to tell your boss that the next product should be developed in Java because you like Java. They want concrete reasons.
- 15y ago
nupark2 15y ago
Given the introduction, I thought we were going to see a fantastic breakdown of cost estimates; capex and opex expenses, 95% percentile bandwidth pricing vs. Amazon's fixed rate per-byte pricing. A real analysis would have even dug into the question of sysadmins vs engineering staff, and the potential for "the cloud" to allow programmers to actually program infrastructure, eliminating internal sysadmin staff completely (and outsourcing all the remaining sysadmin work to Amazon). Instead, we got bizarre platitudes about how they "love computers". How boring. Early in my career, I spent a lot of time in the hot/cold, blaring fan world of data centers -- sometimes late at night, when a hard drive failed or a switch's fans went out. Anyone that thinks that it's pleasant to spend time in a data center is insane; whether or not it makes sense in terms of cost was the answer I'd hoped to get from this article.
- brudgers 15y ago
  It was a real analysis that went beyond just counting the beans. As a privately held company, getting rid of passionate people doesn't make the stock go up, increase anyone's bonus, or benefit their long term goals - and blogging about their values certainly cannot hurt in attracting top notch talent.
  - deleted 15y ago
    [deleted]
  - deleted 15y ago
    [deleted]
  - nupark2 15y ago
    If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste -- and it's impossible to argue against personal taste. An evaluation based on personal taste is useless. If I say "I like orange because it's a great color" -- then either you already like orange, in which case my analysis does you no good, or you don't like orange, in which case my statement can simply be ignored. The same applies here, in that the article could have been coalesced to a single sentence: "We don't use the cloud because we don't want to. We like dorking around with hardware and data centers, just like someone else might link tinkering with their car or collecting bottle caps" Possibly interesting -- but not useful.
quanticle 15y ago
The reason I'm still wary of the cloud is that the abstractions are still too leaky (to borrow Joel Spolsky's turn of phrase). When you start abstracting away core system calls (like fsync), things work great 99.99% of the time. But when that .01% bites you, it bites hard. We don't expect core system calls (like fsync) to fail. And when they fail, the fact that our code is two or three levels levels of abstraction higher means that we often have no way of fixing the issue. The cloud will be at a disadvantage to "real servers" as long as its abstractions are leaky enough to be distinguishable from real hardware.
- wulczer 15y ago
  Cloud or not, if you're writing serious software (and by that I mean software that depends on fsync succeeding) you'd better expect system calls to fail and have code to handle that.
  - ianburrell 15y ago
    The failure with fsync is not that the system call will fail but that it will succeed but be wrong about writing to durable storage.
- wmf 15y ago
  In what way does fsync work differently in the cloud?
  - josephcooney 15y ago
    It seems that (at least for AWS) EBS might not always do what you expect with fsync() http://teddziuba.com/2011/04/amazon-the-purpose-of-pain.html http://teddziuba.com/2011/04/amazon-the-purpose-of-pain.html
  - fleitz 15y ago
    In the way that you have no idea whether fsync actually writes the data to disk, or whether it puts it in memory somewhere (usually a write cache).
lhnn 15y ago
Why is everyone attacking this article? So what if their reason is subjective? They want people who enjoy working on the whole stack. They have fun doing that, and that's all the reason they need. They aren't saying that "people who use the cloud don't care as much". It's just their thing. They're the masters of their own company, and as such, can do things however they like. One of the benefits of a private organization.
- drhayes9 15y ago
  I'm totally with you -- they have fun, that's what they want, end of story. But they did say that people who use the cloud don't care as much: > If you just want to use someone else’s computers, it means you don’t love computers — at least not every aspect to them.
- chollida1 15y ago
  I think the reason most people are disappointed is that from the title they expected the article to be something that it wasnt. I personally thought I going to get a break down of the pros and cons of being cloud hosted vs running your own servers. The article ended up being completely non technical or quantitative and boils down to "we like to build our own computers". Chalk it up to high expectations that weren't met :) That's not the authors fault.
deleted 15y ago
[deleted]
ww520 15y ago
OT: I've heard claims that dedicated servers beat cloud VPS (i.e. AWS) in cost. But most of the dedicated servers I've seen cost $100 and up for a month. Is there a comparison on the cost/computing_power with benchmark for review?
- asharp 15y ago
  What you care about at any reasonable scale (ie. the point at which costs matter) is $/serviced request. That usually ends up being dominated either in $/cpu cycle or $/iop. The current problem is that most cloud platforms suck iops which makes them far more expencive then they really should be. Cloud vs. dedi vs. colo in terms of raw prices though, depends entirely on your price of money and the price you can negotiate with your provider.
Joakal 15y ago
Past article 'Coding Horror: Scaling Up vs. Scaling Out: Hidden Costs' http://www.codinghorror.com/blog/2009/06/scaling-up-vs-scaling-out-hidden-costs.html http://www.codinghorror.com/blog/2009/06/scaling-up-vs-scali... Although it's not quite the cloud, one of the founders prefer vertical stacks due to several reasons including licensing costs.
- asharp 15y ago
  You scale out hardware layer by finding optimal hardware and scaling out on that. The major problem with their 'scale out' chart is that that particular hardware is far from optimal hence costing far more then it should.
espeed 15y ago
One key reason I suspect is that you haven't been able to get SSDs in the cloud so their scaling up approach would be more difficult. Having your own servers means you can "NewEgg your way out of it" (http://news.ycombinator.com/item?id=3243133 http://news.ycombinator.com/item?id=3243133 :-).
jaequery 15y ago
It's because they are a performance hungry company. I thought it was quite obvious by now.
mathattack 15y ago
I think we are overanalyzing here. This is like a car company that maintains a garage to attract people who like to look under the hood. It's taking a strategic people decision when hard data isn't easily available. Everything about sys admin job descriptions seems like a digression.
eduadecastro 15y ago
Indeed the cloud has no soul
david_a_r_kemp 15y ago
I'm growing to understand "the cloud" not as AWS/Rackspace/Heroku/whatever, but as a mindset where: 1) you're not tied to hardware 2) your application scales horizontally 3) you can easily (and dynamically) add (computational) resources as needed, preferrably managed by your application. 4) you build fault tolerance and redundancy into your application This article basically takes the approach that "the cloud" is AWS.

126 points by johns 15y ago

coob 15y ago

So for cultural reasons? Is it not perhaps because there's much less competition for .NET stack cloud platforms, therefore less quality?

droz 15y ago
competition isn't required for quality.
- bprieto 15y ago
  Of course it is. If there is no competition, you can get by with a crappy product or service if your client need it.
  - philwelch 15y ago
    You're arguing that competition is sufficient for quality, not that it's necessary. The first vendor might choose to provide a high quality product regardless, which could, in itself, provide a barrier to entry preventing competition. The real win of competition is that it allows a market to be commoditized, at which point it actually operates like we all learned in our Econ 101 class.
  - hvs 15y ago
    I wouldn't say that it is required, simply that it is more likely to improve quality.
  - Rexxar 15y ago
    I think droz mean "You don't need competition to do a good job" rather than "It's better to have competition when you want buy something of quality". But in either case, sometimes, competition lowers at the same time the price and the quality. So competition doesn't necessarily increase quality.
hughesdan 15y ago
Amazon has very good .NET support. So does Rackspace. So does Azure (obviously). In my personal experience the quality of cloud platforms has been comparable for both Linux and Windows environments. Is there any evidence that .NET cloud platforms are of lower quality? I'm genuinely interested. Please elaborate.
- reuser 15y ago
  Personally, I doubt there is a real quality problem with those platforms. But running .NET on those platforms looks unnecessarily expensive to me. If I were married to Microsoft (as SO is) then I would avoid 'the cloud' for that reason. Given that one wishes to use Windows/.NET, I feel the decision follows pretty naturally from an analysis of costs - no vague argument about culture necessary
  - hughesdan 15y ago
    I would say cost (referring to license fees) is more relevant in deciding whether or not to choose the .NET stack in the first place. You're not going to avoid license fees by avoiding the cloud. Their culture argument is a funny one. But I have no beef with it. If owning and tinkering with the hardware makes them happy, more power to them. As an aside, I've been using Azure lately for my .NET projects. MSFT has done a nice job with that platform and don't get enough credit for it IMHO. I especially like SQL Azure.

MichaelGG 15y ago

What a silly argument. "If you just want to use someone else’s computers, it means you don’t love computers — at least not every aspect to them." Uh, don't they use Dell? If they love them so much, why not build their own? Why stop there? I love software and programming languages, but it doesn't mean I'm going to use my own compiler or run a company on my own libc.

xradionut 15y ago
Whom is going to do a better job? A group of educated, passionate people that can troubleshoot their own problems in realtime? Or "Peggy" at Cloud Vendor X that deals with 400 other clients?
- mechanical_fish 15y ago
  That depends on whether the fix that Peggy deploys for one client also fixes the same problem for the other 399. In other words: It depends. There's no general answer, because deciding where to draw the abstraction barrier between you and your vendors is an unsolved, difficult, and evolving problem.
KyleBrandt 15y ago
There is always the question of degree and practicality. So in terms of libc / and custom compilers that is perhaps ad absurdum. There really wouldn't be any practical benefit taking it to that far for most companies. However, people that love programming languages often probably wish it was practical and that they had an excuse to write compilers at some point. As far as Dell goes, I honestly have mixed feelings at this point. When I started at the company there was less than 10 people, so the idea of having centralized firmware updates etc seemed practical. I'm not sure I really feel this way any more -- I go back forth. But sticking with Dell seems to make sense at this point so we have more uniformity in hardware and management.
- josephcooney 15y ago
  A friend of mine has the uncanny ability of re-formulating every problem he is given in such a way that the logical solution to the problem is always: "Lets write a parser". He loves writing them. Not quite 'writing your own compiler' but it's going that way.

jtbigwoo 15y ago

I suspect that this is part of the problem with many big manufacturing companies these days. Ford was a leader for so many years was because their leadership was passionate about engineering and manufacturing. Henry Ford was an engineer for the Edison Company before he was a manager and is famous for obsessing about every detail of the manufacturing process. That sort of spirit infected the company up through the 1960's and 70's when the MBA's took over and everything was reduced to a column of numbers. It appears that many early tech companies (TI, HP) have headed in the same direction. Maybe 30 - 40 years is the limit before the accountants take over.

mechanical_fish 15y ago
A Ford bought today is less expensive, considerably safer, more efficient, and more reliable than a Ford from the days of Henry Ford. Meanwhile, today's manufacturing companies routinely make objects – like the processor driving the machine that you are reading this on – that are orders of magnitude more complex than the most advanced technology of 1959. And all of this was done by companies that employ lots of accountants. I'm not sure why you think that accountants and engineers don't routinely coexist, or that Henry Ford didn't employ plenty of accountants back in his day. Being able to manufacture good stuff in bulk at reasonable prices is a major exercise in accounting, and always has been.
- Donwangugi 15y ago
  The current CEO of Ford, Alan Mulally is an engineer by trade. I attribute a lot of their current success to him.

jinushaun 15y ago

Poorly written argument full of red herrings. From what I can tell, their answer to the "why aren't you in the cloud" question is that they "love computers so much that they want to do it all themselves because it's so fun".

mechanical_fish 15y ago
Hey, at least they're being honest. Would you have preferred a gigantic stack of technical chaff designed to rationalize and obscure the fact that they're doing whatever seemed like fun at the time? Because I'm sure we can find you some examples. ;)
nephyo 15y ago
If I asked you why you were eating pizza and you responded "because I like pizza" would you still consider that a red herring argument? All they are saying is that they are not in the cloud because they don't want to be in the cloud and because they quite like not being in the cloud. It works for them. Why is that expression at all controversial? If they'd labeled their blog "Why being in the Cloud is a bad idea" or "Why being in the cloud would cost Stack Exchange a lot more" or "Why nobody should ever use the Cloud" then I could see the reason for the dispute. Then their arguments would be very unconvincing. It is certainly valid for Stack Exchange to make a choice because it fits within their culture and then explain that that is why they are doing it. You might think it's a dumb decision and that's fine. But they clearly are not trying to convince you that they've made the best possible decision. They're just saying that good or bad, they've made the decision that they want to make. And it's clearly a decision that has been working well for them so far anyway. So why do people keep bugging them about it?
- jinushaun 15y ago
  You're correct about alternative titles for the blog post. However, the assumption is that on a technical blog, readers would expect technical answers--at least I did. You're not going to tell your boss that the next product should be developed in Java because you like Java. They want concrete reasons.
15y ago

nupark2 15y ago

Given the introduction, I thought we were going to see a fantastic breakdown of cost estimates; capex and opex expenses, 95% percentile bandwidth pricing vs. Amazon's fixed rate per-byte pricing. A real analysis would have even dug into the question of sysadmins vs engineering staff, and the potential for "the cloud" to allow programmers to actually program infrastructure, eliminating internal sysadmin staff completely (and outsourcing all the remaining sysadmin work to Amazon). Instead, we got bizarre platitudes about how they "love computers". How boring. Early in my career, I spent a lot of time in the hot/cold, blaring fan world of data centers -- sometimes late at night, when a hard drive failed or a switch's fans went out. Anyone that thinks that it's pleasant to spend time in a data center is insane; whether or not it makes sense in terms of cost was the answer I'd hoped to get from this article.

brudgers 15y ago
It was a real analysis that went beyond just counting the beans. As a privately held company, getting rid of passionate people doesn't make the stock go up, increase anyone's bonus, or benefit their long term goals - and blogging about their values certainly cannot hurt in attracting top notch talent.
- deleted 15y ago
  [deleted]
- deleted 15y ago
  [deleted]
- nupark2 15y ago
  If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste -- and it's impossible to argue against personal taste. An evaluation based on personal taste is useless. If I say "I like orange because it's a great color" -- then either you already like orange, in which case my analysis does you no good, or you don't like orange, in which case my statement can simply be ignored. The same applies here, in that the article could have been coalesced to a single sentence: "We don't use the cloud because we don't want to. We like dorking around with hardware and data centers, just like someone else might link tinkering with their car or collecting bottle caps" Possibly interesting -- but not useful.

quanticle 15y ago

The reason I'm still wary of the cloud is that the abstractions are still too leaky (to borrow Joel Spolsky's turn of phrase). When you start abstracting away core system calls (like fsync), things work great 99.99% of the time. But when that .01% bites you, it bites hard. We don't expect core system calls (like fsync) to fail. And when they fail, the fact that our code is two or three levels levels of abstraction higher means that we often have no way of fixing the issue. The cloud will be at a disadvantage to "real servers" as long as its abstractions are leaky enough to be distinguishable from real hardware.

wulczer 15y ago
Cloud or not, if you're writing serious software (and by that I mean software that depends on fsync succeeding) you'd better expect system calls to fail and have code to handle that.
- ianburrell 15y ago
  The failure with fsync is not that the system call will fail but that it will succeed but be wrong about writing to durable storage.
wmf 15y ago
In what way does fsync work differently in the cloud?
- josephcooney 15y ago
  It seems that (at least for AWS) EBS might not always do what you expect with fsync() http://teddziuba.com/2011/04/amazon-the-purpose-of-pain.html http://teddziuba.com/2011/04/amazon-the-purpose-of-pain.html
- fleitz 15y ago
  In the way that you have no idea whether fsync actually writes the data to disk, or whether it puts it in memory somewhere (usually a write cache).

lhnn 15y ago

Why is everyone attacking this article? So what if their reason is subjective? They want people who enjoy working on the whole stack. They have fun doing that, and that's all the reason they need. They aren't saying that "people who use the cloud don't care as much". It's just their thing. They're the masters of their own company, and as such, can do things however they like. One of the benefits of a private organization.

drhayes9 15y ago
I'm totally with you -- they have fun, that's what they want, end of story. But they did say that people who use the cloud don't care as much: > If you just want to use someone else’s computers, it means you don’t love computers — at least not every aspect to them.
chollida1 15y ago
I think the reason most people are disappointed is that from the title they expected the article to be something that it wasnt. I personally thought I going to get a break down of the pros and cons of being cloud hosted vs running your own servers. The article ended up being completely non technical or quantitative and boils down to "we like to build our own computers". Chalk it up to high expectations that weren't met :) That's not the authors fault.

deleted 15y ago

[deleted]

ww520 15y ago

OT: I've heard claims that dedicated servers beat cloud VPS (i.e. AWS) in cost. But most of the dedicated servers I've seen cost $100 and up for a month. Is there a comparison on the cost/computing_power with benchmark for review?

asharp 15y ago
What you care about at any reasonable scale (ie. the point at which costs matter) is $/serviced request. That usually ends up being dominated either in $/cpu cycle or $/iop. The current problem is that most cloud platforms suck iops which makes them far more expencive then they really should be. Cloud vs. dedi vs. colo in terms of raw prices though, depends entirely on your price of money and the price you can negotiate with your provider.

Joakal 15y ago

Past article 'Coding Horror: Scaling Up vs. Scaling Out: Hidden Costs' http://www.codinghorror.com/blog/2009/06/scaling-up-vs-scaling-out-hidden-costs.html http://www.codinghorror.com/blog/2009/06/scaling-up-vs-scali... Although it's not quite the cloud, one of the founders prefer vertical stacks due to several reasons including licensing costs.

asharp 15y ago
You scale out hardware layer by finding optimal hardware and scaling out on that. The major problem with their 'scale out' chart is that that particular hardware is far from optimal hence costing far more then it should.

espeed 15y ago

One key reason I suspect is that you haven't been able to get SSDs in the cloud so their scaling up approach would be more difficult. Having your own servers means you can "NewEgg your way out of it" (http://news.ycombinator.com/item?id=3243133 http://news.ycombinator.com/item?id=3243133 :-).

jaequery 15y ago

It's because they are a performance hungry company. I thought it was quite obvious by now.

mathattack 15y ago

I think we are overanalyzing here. This is like a car company that maintains a garage to attract people who like to look under the hood. It's taking a strategic people decision when hard data isn't easily available. Everything about sys admin job descriptions seems like a digression.

eduadecastro 15y ago

Indeed the cloud has no soul

david_a_r_kemp 15y ago

I'm growing to understand "the cloud" not as AWS/Rackspace/Heroku/whatever, but as a mindset where: 1) you're not tied to hardware 2) your application scales horizontally 3) you can easily (and dynamically) add (computational) resources as needed, preferrably managed by your application. 4) you build fault tolerance and redundancy into your application This article basically takes the approach that "the cloud" is AWS.