3 ms·

The cost starts out too high to justify releasing something that's not optimized when you're operating at Google's scale, especially since the development costs

The cost starts out too high to justify releasing something that's not optimized when you're operating at Google's scale, especially since the development costs are largely up front whereas the ongoing compute costs keep going for at least a few years. If they didn't keep an eye on this stuff, those factors of 2-10x would eat them alive. For some value of M and N, having N million users around for M months means that development costs are actually cheaper than compute costs, and I'd assume that they've now reached a point where most new products expect to see more than that magic number of users. A few engineers for a few extra months is only ~100k, and I have no trouble believing that many Google products cost them at least that much over their lifetimes due to resource consumption. The rest of us can safely ignore all this because our products actually need to grow before we pass that threshold, at which point we deal with the issues; Google is in an enviable position where no optimization is premature. Even for the rest of us, though, given the performance differences in Python vs. Java (best case 2-4x, worst case more like 40x) relative to the productivity differences (I can't believe this is more than 10x, even for someone very comfortable in Python and merely competent at Java), I'd suspect that many high use software projects even outside of behemoths like Google are actually cheaper in the long run if they're done in Java than they would be in Python. Prototypes are another issue altogether, but I haven't seen anything that says Google is discouraging people to do those in whatever language they want; AFAIK it's production code that we're talking about here.

btilly 17y ago
Some concrete numbers on the productivity differences would help. The chart in Software Estimation by McConnell, adapted from Software Cost Estimation with Cocomo II is that projects are 2.5 * bigger in C than Java, and 6 * bigger in C than they would be in Perl or Smalltalk. Assuming that Python is equivalent to Perl, that would mean that Java requires 2.4 times as many lines. Those estimates are from 2000. Both languages have improved since then, but I'll go with that estimate because I don't have more recent figures backed up by quantitative data rather than someone's opinion. Research across many languages suggests that lines of code/developer/day are roughly constant, so Python development should average 2.4 times as fast as Java. Let's suppose that your programmers are paid $80K/year. On average people cost about double their salary (after you include benefits, tools, office space, etc), so each programmer costs you $160K/year. Per Python programmer replaced you need 2.4 Java programmers which means an extra $224K/year in cost. Let's suppose your computers have an operating cost of $14K/year. (I am throwing out a figure for regular replacement, networking, electricity, sysadmins, etc.) Then the cost per Python programmer replaced will cover 16 more machines. And it can get worse. Research into productivity versus group size suggests that productivity peaks at about 5-7 people. When you have more developers than that the overhead for communication exceeds productive work done unless you introduce processes to limit direct communication. Those processes themselves reduce productivity. As a result you don't get back to the same productivity a team of 5-7 people has until you have a team of around 20 people. If you grow you'll eventually need to make this transition, but it should be left as long as possible. Therefore if a Python shop currently has less than a dozen webservers per developer, switching to Java does not look like it makes sense. And furthermore if you've got 3-7 Python developers then the numbers suggest that a Java switch will force you over the maximum small team size and force you to wind up with a large team at much higher expense. For this reason I believe that the vast majority of companies using agile languages like Perl, Ruby and Python would be worse off if they switched to Java. When you serve traffic at the scale of Google this dynamic changes. But most of us aren't Google.
- nearestneighbor 17y ago
  > Research across many languages suggests that lines of code/developer/day are roughly constant, so Python development should average 2.4 times as fast as Java. Some people, with more Java experience than me, claim that modern IDEs like IntelliJ make them as productive with Java as they would be with Python. Java's verbosity is "mechanical", and if your tools help you with that, it's hard for me to buy 2.4x productivity difference. You are also assuming that static typing has no effect on how many people can work together. In Python (which I use for prototyping and appreciate), one might often wonder "does this method take a class, an instance of a class, or the returned value as an argument", etc.
  - btilly 17y ago
    Debates on relative productivity are endless. I used the only numbers I have available that are backed by actual quantitative data rather than opinion. I'd love to see more up to date numbers. However even if you assume that the difference is much smaller, say a factor of 1.2, then you can afford an extra 2.3 computers for every developer. Oddly enough in the various companies I know well with small teams of experienced scripting programmers I've never seen that high a ratio of webservers to developers, so even so switching to Java doesn't make sense. On the question how many people can work together, needing to talk about data types is such a small portion of what people talk about that I would be shocked if it changes where the cutoff is between where small teams break down, or where large teams become as productive as that peak. That said I fully agree that Java is designed to let large teams cooperate, and that's likely to matter when you have teams of 50+ programmers. However if the productivity difference really is a factor of 2.4, and we assume linear growth in productivity for large teams, then you'll actually need a team of 50 or so programmers in Java to match the team of 5-7 programmers working in a modern scripting language. Given that, if you're working in a Java team below that size you should seriously ask yourself whether having a team size that requires getting that many people working together is a self-inflicted problem.

btilly 17y ago

Some concrete numbers on the productivity differences would help. The chart in Software Estimation by McConnell, adapted from Software Cost Estimation with Cocomo II is that projects are 2.5 * bigger in C than Java, and 6 * bigger in C than they would be in Perl or Smalltalk. Assuming that Python is equivalent to Perl, that would mean that Java requires 2.4 times as many lines. Those estimates are from 2000. Both languages have improved since then, but I'll go with that estimate because I don't have more recent figures backed up by quantitative data rather than someone's opinion. Research across many languages suggests that lines of code/developer/day are roughly constant, so Python development should average 2.4 times as fast as Java. Let's suppose that your programmers are paid $80K/year. On average people cost about double their salary (after you include benefits, tools, office space, etc), so each programmer costs you $160K/year. Per Python programmer replaced you need 2.4 Java programmers which means an extra $224K/year in cost. Let's suppose your computers have an operating cost of $14K/year. (I am throwing out a figure for regular replacement, networking, electricity, sysadmins, etc.) Then the cost per Python programmer replaced will cover 16 more machines. And it can get worse. Research into productivity versus group size suggests that productivity peaks at about 5-7 people. When you have more developers than that the overhead for communication exceeds productive work done unless you introduce processes to limit direct communication. Those processes themselves reduce productivity. As a result you don't get back to the same productivity a team of 5-7 people has until you have a team of around 20 people. If you grow you'll eventually need to make this transition, but it should be left as long as possible. Therefore if a Python shop currently has less than a dozen webservers per developer, switching to Java does not look like it makes sense. And furthermore if you've got 3-7 Python developers then the numbers suggest that a Java switch will force you over the maximum small team size and force you to wind up with a large team at much higher expense. For this reason I believe that the vast majority of companies using agile languages like Perl, Ruby and Python would be worse off if they switched to Java. When you serve traffic at the scale of Google this dynamic changes. But most of us aren't Google.

nearestneighbor 17y ago
> Research across many languages suggests that lines of code/developer/day are roughly constant, so Python development should average 2.4 times as fast as Java. Some people, with more Java experience than me, claim that modern IDEs like IntelliJ make them as productive with Java as they would be with Python. Java's verbosity is "mechanical", and if your tools help you with that, it's hard for me to buy 2.4x productivity difference. You are also assuming that static typing has no effect on how many people can work together. In Python (which I use for prototyping and appreciate), one might often wonder "does this method take a class, an instance of a class, or the returned value as an argument", etc.
- btilly 17y ago
  Debates on relative productivity are endless. I used the only numbers I have available that are backed by actual quantitative data rather than opinion. I'd love to see more up to date numbers. However even if you assume that the difference is much smaller, say a factor of 1.2, then you can afford an extra 2.3 computers for every developer. Oddly enough in the various companies I know well with small teams of experienced scripting programmers I've never seen that high a ratio of webservers to developers, so even so switching to Java doesn't make sense. On the question how many people can work together, needing to talk about data types is such a small portion of what people talk about that I would be shocked if it changes where the cutoff is between where small teams break down, or where large teams become as productive as that peak. That said I fully agree that Java is designed to let large teams cooperate, and that's likely to matter when you have teams of 50+ programmers. However if the productivity difference really is a factor of 2.4, and we assume linear growth in productivity for large teams, then you'll actually need a team of 50 or so programmers in Java to match the team of 5-7 programmers working in a modern scripting language. Given that, if you're working in a Java team below that size you should seriously ask yourself whether having a team size that requires getting that many people working together is a self-inflicted problem.