6 ms·

> If you’re writing Ruby code to turn your blog drafts into HTML, it doesn’t much matter if republishing takes a few extra seconds Unless your software is inte

> If you’re writing Ruby code to turn your blog drafts into HTML, it doesn’t much matter if republishing takes a few extra seconds Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators and it takes ~5,000 ms on an i9 machine when it could take 100 ms if different languages and different implementation choices were made. This is the story of modern software: "I don't care how much time and computing power I waste. It's not the core app of some FAANG giant, so I'm not going to bother, ever."

bluedino 4y ago
It's times like these when we forget Turbo Pascal compiling 40,000 lines per second on a 386
dahfizz 4y ago
I wonder how heavily other fields sacrifice in the name of "ease of implementation". Could we have houses that are 100x stronger and longer lasting if we allowed a few extra weeks of construction time? Could we 10x battery capacity with a slightly more sophisticated manufacturing process? I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software.
- ajmurmann 4y ago
  For the economics of that comparison to make sense, we should also include material inputs to the construction. It doesn't matter to the cost of the final product if it was materials that were saved or labor cost. Software is the special case where cost almost entirely comes down to labor. So given that, we are of course constantly compromising quality. We realize that less though because buildings are more standardized.
- TimPC 4y ago
  The big issue in other fields mostly isn't manufacturing design or construction process. The main bottleneck is with physical goods and long distances travelled you're subject to high shipping costs so people cut all sorts of corners to make things less bulky and lighter. Cheap plastics are far lighter than wood or metals for instance so we see more plastics get used.
- adamdusty 4y ago
  I work in III/V semiconductors. The product development goes like: identify need, design, attempt a proof of concept, DoEs to determine manufacturing costs/viability, repeat until you have a statistically controlled viable process. There is essentially no room for technical creativity like there is in software. If we spent an extra year on our worst performing product (we've done this), we would get at best 5% improvement with iffy reproducibility. I dont know about construction or batteries.
- astrange 4y ago
  You could have a lot more good enough and much cheaper houses in the US, but we banned manufactured/prefab houses due to lobbying from the “build everything individually out of wood” lobby, require huge setbacks due to the front lawn lobby, and various other things like overly wide roads because of out of date fire codes.
- mr_toad 4y ago
  > I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software. Most, as in > 50%, of developers aren’t in a position to decide which languages, libraries and tools to use, or how much time they can spend on performance improvement.
- Gigachad 4y ago
  Anyone with APM setup can see what’s going on pretty well. For almost all web apps, you can see that ruby/python are extremely fast and almost all of the time spent on a request is the database.
  - dgb23 4y ago
    The implication is likely that the db does all the work, and the parts in ruby/python have barely anything to do.
    - Gigachad 4y ago
      This is true of every rails app I have worked on in the last 10 years. You’d have to be doing something remarkably bad or have some crazy situation to end up with ruby using any noticeable amount of compute time.
      - dgb23 4y ago
        Well that depends on what your application is doing. But yes, if it is a database heavy workload, then what you say applies. What I was trying to get at was that in these cases the ruby/python/whatever parts are not actually fast, they just do much less stuff. Add that the database might be on another machine, so there's plenty of overhead as well. Aside: You can sometimes get orders of magnitude of speed gains just by using an embedded database and a fast language on top if your workload allows it. Essentially you don't really do much to get this, just cut overhead.
dwrodri 4y ago
I agree to the extent that people fail to realize how they could be missing out opportunities to innovate or corner a market when they leave performance on the table. Quite often, new products become possible when a basic task like rendering HTML goes from taking 10 seconds to 10ms. I think you can paint the problem in broad strokes from either side of the "how important is performance?" argument. From a "get the bills paid" point-of-view, any good project manager also has to know when to tell an engineer to focus on getting the product shipped instead of chasing that next 5% in throughput/latency reduction/overhead. I've seen my fair share of programmers (including myself) refuse to ship to a project because the pursuit of some "undesirable" latency and not finish more important features. For tasks like video streaming, Automation software (CI pipelines to robotics), video games, professional tools for content creation (DAWs, video editing, Blender, etc.) performance is the feature, but then your product is helping them get the bills paid faster. Medical apparatus(es?) and guidance software on autonomous vehicles are examples of where latency is a life-or-death situation. I think everyone would benefit from playing with OpenRTOS, or writing some code that deals with video/audio where there are hard deadlines on latency. But I'm never gonna hold some weekend-project static site generator in Ruby to the same standard as macOS.
- makapuf 4y ago
  Agreed with what you said, but even for Blender, performance is important but the feature are Free software, good modeler, correct, feature-packed, good looking renderer AND performance.
agumonkey 4y ago
aws recently blogged about price benefits of using more performant compiled languages on their servers, it's coming
Jtsummers 4y ago
Probably why he had the next paragraph: > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers, it makes sense to think hard about performance at design time. Your scenario falls under that category.
- anonymoushn 4y ago
  It's the other way around right? If you think about performance, you won't need to orchestrate a fleet.
- _gabe_ 4y ago
  > Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators This falls into this? > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers A static site generator is a simple utility that should take 100ms at most on a modern crappy laptop. It's not some backend service that's in a critical path, but it's also not a difficult engineering task. Parse input, produce output. This isn't something that should take seconds, which I think is what the OP was getting at. But because of the language choices made, and the millions of lines of bloated code, it does take seconds.
  - Jtsummers 4y ago
    Yes. Maybe not directly, but more like it versus a one-off "I just made a new blog post and can wait 10 seconds for it to render and deploy." If you're reloading the blog posts live then you're on the critical path, not a one-off anymore. So you need to think about performance. In the former case, the performance really doesn't matter. If you've got a personal blog, does it matter if your update takes 10 seconds or 1 second? Probably not, if it does it's the most important blog in the world. If you've got customers with many blogs and need to take their updates and render them, then the performance matters because it's shifted from 1 to many (hundreds? thousands?). And now that 10 second delay is a big issue, you're either using a fleet of servers to handle the load or some customers don't see updates for days (oops).
paganel 4y ago
It was the "bad" untyped languages like PHP, Python and Ruby (Perl was too complicated for us, mere mortals) that saved the web from becoming a Microsoft monopoly, or, more likely, an oligopoly between the same MS, probably Sun, probably IBM or some such. I'm talking about most of the 2000s decade. True, the web has become sort of an oligopoly right now, but at least that was not caused by the programming languages and web frameworks that power it. What I'm trying to say is that those languages that everyone is quick to judge right now have given us 10, maybe 15 years of extra "life", a period when most of us have "made" our careers and, well, most of our money (those who have managed to make that money, that is). We wouldn't have had (what basically are) web companies worth tens, if not hundreds of billions of dollars, if the web had still meant relying on Struts or on whatever it is Microsoft was putting forward as a web framework in the mid-2000s. We wouldn't have had engineers taking home TC worth 500-600k and then complaining that Python or Ruby are not what the world needs.
- rentfree-media 4y ago
  But you might have a viable visual page builder. It's honestly a tough choice at times...
  - j-james 4y ago
    BlueGriffon is pretty good.
- pdimitar 4y ago
  Interesting claim, any written proof to back it up?
  - paganel 4y ago
    It would be nice to have some University poor souls study that period from a sociological pov (and not only) but, helas, afaik nobody is doing that so that scattered comments like mine, from people who have lived through that period, is the only thing that we have. Come to think of it maybe some data historians at Google can look through the Googlebot logs and not only and compare the percent of websites that they think were .php-run (let’s say) over all this period, in both nominal and relative terms. It is my hunch that those websites represented a big chunk of Googlebot’s hit targets for most of Google’s (the company) existence, hence why I think my comment is valid (among other things)

bluedino 4y ago

It's times like these when we forget Turbo Pascal compiling 40,000 lines per second on a 386

dahfizz 4y ago

I wonder how heavily other fields sacrifice in the name of "ease of implementation". Could we have houses that are 100x stronger and longer lasting if we allowed a few extra weeks of construction time? Could we 10x battery capacity with a slightly more sophisticated manufacturing process? I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software.

ajmurmann 4y ago
For the economics of that comparison to make sense, we should also include material inputs to the construction. It doesn't matter to the cost of the final product if it was materials that were saved or labor cost. Software is the special case where cost almost entirely comes down to labor. So given that, we are of course constantly compromising quality. We realize that less though because buildings are more standardized.
TimPC 4y ago
The big issue in other fields mostly isn't manufacturing design or construction process. The main bottleneck is with physical goods and long distances travelled you're subject to high shipping costs so people cut all sorts of corners to make things less bulky and lighter. Cheap plastics are far lighter than wood or metals for instance so we see more plastics get used.
adamdusty 4y ago
I work in III/V semiconductors. The product development goes like: identify need, design, attempt a proof of concept, DoEs to determine manufacturing costs/viability, repeat until you have a statistically controlled viable process. There is essentially no room for technical creativity like there is in software. If we spent an extra year on our worst performing product (we've done this), we would get at best 5% improvement with iffy reproducibility. I dont know about construction or batteries.
astrange 4y ago
You could have a lot more good enough and much cheaper houses in the US, but we banned manufactured/prefab houses due to lobbying from the “build everything individually out of wood” lobby, require huge setbacks due to the front lawn lobby, and various other things like overly wide roads because of out of date fire codes.
mr_toad 4y ago
> I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software. Most, as in > 50%, of developers aren’t in a position to decide which languages, libraries and tools to use, or how much time they can spend on performance improvement.
Gigachad 4y ago
Anyone with APM setup can see what’s going on pretty well. For almost all web apps, you can see that ruby/python are extremely fast and almost all of the time spent on a request is the database.
- dgb23 4y ago
  The implication is likely that the db does all the work, and the parts in ruby/python have barely anything to do.
  - Gigachad 4y ago
    This is true of every rails app I have worked on in the last 10 years. You’d have to be doing something remarkably bad or have some crazy situation to end up with ruby using any noticeable amount of compute time.
    - dgb23 4y ago
      Well that depends on what your application is doing. But yes, if it is a database heavy workload, then what you say applies. What I was trying to get at was that in these cases the ruby/python/whatever parts are not actually fast, they just do much less stuff. Add that the database might be on another machine, so there's plenty of overhead as well. Aside: You can sometimes get orders of magnitude of speed gains just by using an embedded database and a fast language on top if your workload allows it. Essentially you don't really do much to get this, just cut overhead.

dwrodri 4y ago

I agree to the extent that people fail to realize how they could be missing out opportunities to innovate or corner a market when they leave performance on the table. Quite often, new products become possible when a basic task like rendering HTML goes from taking 10 seconds to 10ms. I think you can paint the problem in broad strokes from either side of the "how important is performance?" argument. From a "get the bills paid" point-of-view, any good project manager also has to know when to tell an engineer to focus on getting the product shipped instead of chasing that next 5% in throughput/latency reduction/overhead. I've seen my fair share of programmers (including myself) refuse to ship to a project because the pursuit of some "undesirable" latency and not finish more important features. For tasks like video streaming, Automation software (CI pipelines to robotics), video games, professional tools for content creation (DAWs, video editing, Blender, etc.) performance is the feature, but then your product is helping them get the bills paid faster. Medical apparatus(es?) and guidance software on autonomous vehicles are examples of where latency is a life-or-death situation. I think everyone would benefit from playing with OpenRTOS, or writing some code that deals with video/audio where there are hard deadlines on latency. But I'm never gonna hold some weekend-project static site generator in Ruby to the same standard as macOS.

makapuf 4y ago
Agreed with what you said, but even for Blender, performance is important but the feature are Free software, good modeler, correct, feature-packed, good looking renderer AND performance.

agumonkey 4y ago

aws recently blogged about price benefits of using more performant compiled languages on their servers, it's coming

Jtsummers 4y ago

Probably why he had the next paragraph: > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers, it makes sense to think hard about performance at design time. Your scenario falls under that category.

anonymoushn 4y ago
It's the other way around right? If you think about performance, you won't need to orchestrate a fleet.
_gabe_ 4y ago
> Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators This falls into this? > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers A static site generator is a simple utility that should take 100ms at most on a modern crappy laptop. It's not some backend service that's in a critical path, but it's also not a difficult engineering task. Parse input, produce output. This isn't something that should take seconds, which I think is what the OP was getting at. But because of the language choices made, and the millions of lines of bloated code, it does take seconds.
- Jtsummers 4y ago
  Yes. Maybe not directly, but more like it versus a one-off "I just made a new blog post and can wait 10 seconds for it to render and deploy." If you're reloading the blog posts live then you're on the critical path, not a one-off anymore. So you need to think about performance. In the former case, the performance really doesn't matter. If you've got a personal blog, does it matter if your update takes 10 seconds or 1 second? Probably not, if it does it's the most important blog in the world. If you've got customers with many blogs and need to take their updates and render them, then the performance matters because it's shifted from 1 to many (hundreds? thousands?). And now that 10 second delay is a big issue, you're either using a fleet of servers to handle the load or some customers don't see updates for days (oops).

paganel 4y ago

It was the "bad" untyped languages like PHP, Python and Ruby (Perl was too complicated for us, mere mortals) that saved the web from becoming a Microsoft monopoly, or, more likely, an oligopoly between the same MS, probably Sun, probably IBM or some such. I'm talking about most of the 2000s decade. True, the web has become sort of an oligopoly right now, but at least that was not caused by the programming languages and web frameworks that power it. What I'm trying to say is that those languages that everyone is quick to judge right now have given us 10, maybe 15 years of extra "life", a period when most of us have "made" our careers and, well, most of our money (those who have managed to make that money, that is). We wouldn't have had (what basically are) web companies worth tens, if not hundreds of billions of dollars, if the web had still meant relying on Struts or on whatever it is Microsoft was putting forward as a web framework in the mid-2000s. We wouldn't have had engineers taking home TC worth 500-600k and then complaining that Python or Ruby are not what the world needs.

rentfree-media 4y ago
But you might have a viable visual page builder. It's honestly a tough choice at times...
- j-james 4y ago
  BlueGriffon is pretty good.
pdimitar 4y ago
Interesting claim, any written proof to back it up?
- paganel 4y ago
  It would be nice to have some University poor souls study that period from a sociological pov (and not only) but, helas, afaik nobody is doing that so that scattered comments like mine, from people who have lived through that period, is the only thing that we have. Come to think of it maybe some data historians at Google can look through the Googlebot logs and not only and compare the percent of websites that they think were .php-run (let’s say) over all this period, in both nominal and relative terms. It is my hunch that those websites represented a big chunk of Googlebot’s hit targets for most of Google’s (the company) existence, hence why I think my comment is valid (among other things)