5 ms·

If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste -- and it's impos

by nupark2 15y ago

If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste -- and it's impossible to argue against personal taste. An evaluation based on personal taste is useless. If I say "I like orange because it's a great color" -- then either you already like orange, in which case my analysis does you no good, or you don't like orange, in which case my statement can simply be ignored. The same applies here, in that the article could have been coalesced to a single sentence: "We don't use the cloud because we don't want to. We like dorking around with hardware and data centers, just like someone else might link tinkering with their car or collecting bottle caps" Possibly interesting -- but not useful.

KyleBrandt 15y ago
Okay, I'll bite... > I spent a lot of time in the hot/cold, blaring fan world of data centers -- sometimes late at night, when a hard drive failed or a switch's fans went out. If a hard drive or a fan failing requires someone's attention at the data center in the middle of the night something is wrong (unless your company has such high uptime requirements that even redundant systems need to be fixed right away). Either your company can't afford to provide proper redundancy, or someone on the technical side has failed to implement redundancy or failover properly (which happens, but should be corrected). For example we had an entire router fail and this just caused a blip. We didn't have to do anything right away to fix it, just figure out what went wrong and bring it back up the next day. Also we don't really end up spending very much time in the data center itself (in fact I work remotely full time). > We like dorking around with hardware and data centers, just like someone else might link tinkering with their car or collecting bottle caps We do like to consider ourselves experts -- we don't just dork around and tinker (at least, not most of the time) > If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste In part this is a matter of taste. But as I said: "This culture means when we hire technical staff, we hire people who share this passion. I believe that this passion translates into a better product. Whenever someone does a cost analysis of cloud vs self hosting there is no row in the spreadsheet for “Work Productivity Increase due to Passion.” Just because there is no good method to put it on a spreadsheet, doesn't mean it doesn't have value for the company. There are a lot of technical reasons we like to have control over the full stack. But I think it largely comes down to our culture. The culture of a company is very important. A culture could be all about how the numbers fall after you calculate your capital and operating expenses etc. Somehow it seems to me though that many people don't seem very happy in those cultures -- and this will effect employee retention and the sort of talent you can attract.
- mwsherman 15y ago
  For me, the best justification for “doing it ourselves” is that it’s what exceptional companies do. We dig under abstraction layers not knowing what we’ll find. We dig under Linq to SQL. We dig under Redis clients. We really never consider the vendor’s platform good enough. The result of a million small optimizations is a platform that others can’t create.
- kamaal 15y ago
  Thanks for the honest reply. But come to think of it in an another way. If you actually let some one else focus on the issues you don't necessarily need to handle you can focus well on issue you actually need to work on. But, yes its all about personal preferences and I respect your preferences. In my case I would actually let some one else do the job I don't necessarily have to do and use the same time to do well in my actual job.
ssmoot 15y ago
Instead of 12 physical cores, 96GB of RAM and a 2TB SSD Array pushing 1M IOPs on dedicated hardware for my PostgreSQL database servers, I'd need 1TB of RAM in an AWS box because I'll be lucky if I can even break 10K in IOPs. Does the price make sense then? I have yet to see any significant AWS deployment that doesn't feel like it could be done better, more reliably, and much more cheaply as a co-located setup.
- reuser 15y ago
  You can't really create a massive co-located setup on demand for big jobs, then tear it down ... use the right tools for whatever job you're doing. EC2 being more expensive for your Postgres deploy doesn't mean it is an expensive toy
  - ssmoot 15y ago
    I don't think I could say it better: http://twitter.com/#!/dhh/status/133656040561577984 http://twitter.com/#!/dhh/status/133656040561577984 If you really have a need to setup and teardown a bunch of cores for the occassional large batch, nobody's questioning that the cloud is an economical way to do that. But I've never had to do that. Not in a way that would make the development time involved economical anyway. Wait two hours for this once-in-a-blue-moon processing job to finish, or spend a day setting up a process to handle such jobs quickly in the future? It'd really take something exceptional (again, with low IOPs demands) to have that make much sense unless I was already hosting in the cloud and had invested money in making such a task quick and cheap.
- asharp 15y ago
  Saying that you can't get IOPS on AWS is like saying that your machinegun sucks at grating cheese. It's really not meant to do that. But that's fine as AWS isn't the only cloud platform out there, and some are actually desgined to take data intensive workloads.
  - ericd 15y ago
    The problem is that it's touted as a great way to scale web apps. Since DB performance is usually the limiting factor in webapp scaling, this doesn't appear to stack up. Needing a lot of IOPS is pretty par for the course.

by nupark2 15y ago

KyleBrandt 15y ago

Okay, I'll bite... > I spent a lot of time in the hot/cold, blaring fan world of data centers -- sometimes late at night, when a hard drive failed or a switch's fans went out. If a hard drive or a fan failing requires someone's attention at the data center in the middle of the night something is wrong (unless your company has such high uptime requirements that even redundant systems need to be fixed right away). Either your company can't afford to provide proper redundancy, or someone on the technical side has failed to implement redundancy or failover properly (which happens, but should be corrected). For example we had an entire router fail and this just caused a blip. We didn't have to do anything right away to fix it, just figure out what went wrong and bring it back up the next day. Also we don't really end up spending very much time in the data center itself (in fact I work remotely full time). > We like dorking around with hardware and data centers, just like someone else might link tinkering with their car or collecting bottle caps We do like to consider ourselves experts -- we don't just dork around and tinker (at least, not most of the time) > If you remove any rational analysis based on productivity, developer time, or finances, then we're only left with a question of personal taste In part this is a matter of taste. But as I said: "This culture means when we hire technical staff, we hire people who share this passion. I believe that this passion translates into a better product. Whenever someone does a cost analysis of cloud vs self hosting there is no row in the spreadsheet for “Work Productivity Increase due to Passion.” Just because there is no good method to put it on a spreadsheet, doesn't mean it doesn't have value for the company. There are a lot of technical reasons we like to have control over the full stack. But I think it largely comes down to our culture. The culture of a company is very important. A culture could be all about how the numbers fall after you calculate your capital and operating expenses etc. Somehow it seems to me though that many people don't seem very happy in those cultures -- and this will effect employee retention and the sort of talent you can attract.

mwsherman 15y ago
For me, the best justification for “doing it ourselves” is that it’s what exceptional companies do. We dig under abstraction layers not knowing what we’ll find. We dig under Linq to SQL. We dig under Redis clients. We really never consider the vendor’s platform good enough. The result of a million small optimizations is a platform that others can’t create.
kamaal 15y ago
Thanks for the honest reply. But come to think of it in an another way. If you actually let some one else focus on the issues you don't necessarily need to handle you can focus well on issue you actually need to work on. But, yes its all about personal preferences and I respect your preferences. In my case I would actually let some one else do the job I don't necessarily have to do and use the same time to do well in my actual job.

ssmoot 15y ago

Instead of 12 physical cores, 96GB of RAM and a 2TB SSD Array pushing 1M IOPs on dedicated hardware for my PostgreSQL database servers, I'd need 1TB of RAM in an AWS box because I'll be lucky if I can even break 10K in IOPs. Does the price make sense then? I have yet to see any significant AWS deployment that doesn't feel like it could be done better, more reliably, and much more cheaply as a co-located setup.

reuser 15y ago
You can't really create a massive co-located setup on demand for big jobs, then tear it down ... use the right tools for whatever job you're doing. EC2 being more expensive for your Postgres deploy doesn't mean it is an expensive toy
- ssmoot 15y ago
  I don't think I could say it better: http://twitter.com/#!/dhh/status/133656040561577984 http://twitter.com/#!/dhh/status/133656040561577984 If you really have a need to setup and teardown a bunch of cores for the occassional large batch, nobody's questioning that the cloud is an economical way to do that. But I've never had to do that. Not in a way that would make the development time involved economical anyway. Wait two hours for this once-in-a-blue-moon processing job to finish, or spend a day setting up a process to handle such jobs quickly in the future? It'd really take something exceptional (again, with low IOPs demands) to have that make much sense unless I was already hosting in the cloud and had invested money in making such a task quick and cheap.
asharp 15y ago
Saying that you can't get IOPS on AWS is like saying that your machinegun sucks at grating cheese. It's really not meant to do that. But that's fine as AWS isn't the only cloud platform out there, and some are actually desgined to take data intensive workloads.
- ericd 15y ago
  The problem is that it's touted as a great way to scale web apps. Since DB performance is usually the limiting factor in webapp scaling, this doesn't appear to stack up. Needing a lot of IOPS is pretty par for the course.