9 ms·

Making Code Faster

149 points by zdw 4y ago

henning 4y ago
> If you’re writing Ruby code to turn your blog drafts into HTML, it doesn’t much matter if republishing takes a few extra seconds Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators and it takes ~5,000 ms on an i9 machine when it could take 100 ms if different languages and different implementation choices were made. This is the story of modern software: "I don't care how much time and computing power I waste. It's not the core app of some FAANG giant, so I'm not going to bother, ever."
- bluedino 4y ago
  It's times like these when we forget Turbo Pascal compiling 40,000 lines per second on a 386
- dahfizz 4y ago
  I wonder how heavily other fields sacrifice in the name of "ease of implementation". Could we have houses that are 100x stronger and longer lasting if we allowed a few extra weeks of construction time? Could we 10x battery capacity with a slightly more sophisticated manufacturing process? I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software.
  - ajmurmann 4y ago
    For the economics of that comparison to make sense, we should also include material inputs to the construction. It doesn't matter to the cost of the final product if it was materials that were saved or labor cost. Software is the special case where cost almost entirely comes down to labor. So given that, we are of course constantly compromising quality. We realize that less though because buildings are more standardized.
  - TimPC 4y ago
    The big issue in other fields mostly isn't manufacturing design or construction process. The main bottleneck is with physical goods and long distances travelled you're subject to high shipping costs so people cut all sorts of corners to make things less bulky and lighter. Cheap plastics are far lighter than wood or metals for instance so we see more plastics get used.
  - adamdusty 4y ago
    I work in III/V semiconductors. The product development goes like: identify need, design, attempt a proof of concept, DoEs to determine manufacturing costs/viability, repeat until you have a statistically controlled viable process. There is essentially no room for technical creativity like there is in software. If we spent an extra year on our worst performing product (we've done this), we would get at best 5% improvement with iffy reproducibility. I dont know about construction or batteries.
  - astrange 4y ago
    You could have a lot more good enough and much cheaper houses in the US, but we banned manufactured/prefab houses due to lobbying from the “build everything individually out of wood” lobby, require huge setbacks due to the front lawn lobby, and various other things like overly wide roads because of out of date fire codes.
  - mr_toad 4y ago
    > I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software. Most, as in > 50%, of developers aren’t in a position to decide which languages, libraries and tools to use, or how much time they can spend on performance improvement.
  - Gigachad 4y ago
    Anyone with APM setup can see what’s going on pretty well. For almost all web apps, you can see that ruby/python are extremely fast and almost all of the time spent on a request is the database.
    - dgb23 4y ago
      The implication is likely that the db does all the work, and the parts in ruby/python have barely anything to do.
      - Gigachad 4y ago
        This is true of every rails app I have worked on in the last 10 years. You’d have to be doing something remarkably bad or have some crazy situation to end up with ruby using any noticeable amount of compute time.
        dgb23 4y ago
        Well that depends on what your application is doing. But yes, if it is a database heavy workload, then what you say applies. What I was trying to get at was that in these cases the ruby/python/whatever parts are not actually fast, they just do much less stuff. Add that the database might be on another machine, so there's plenty of overhead as well. Aside: You can sometimes get orders of magnitude of speed gains just by using an embedded database and a fast language on top if your workload allows it. Essentially you don't really do much to get this, just cut overhead.
- dwrodri 4y ago
  I agree to the extent that people fail to realize how they could be missing out opportunities to innovate or corner a market when they leave performance on the table. Quite often, new products become possible when a basic task like rendering HTML goes from taking 10 seconds to 10ms. I think you can paint the problem in broad strokes from either side of the "how important is performance?" argument. From a "get the bills paid" point-of-view, any good project manager also has to know when to tell an engineer to focus on getting the product shipped instead of chasing that next 5% in throughput/latency reduction/overhead. I've seen my fair share of programmers (including myself) refuse to ship to a project because the pursuit of some "undesirable" latency and not finish more important features. For tasks like video streaming, Automation software (CI pipelines to robotics), video games, professional tools for content creation (DAWs, video editing, Blender, etc.) performance is the feature, but then your product is helping them get the bills paid faster. Medical apparatus(es?) and guidance software on autonomous vehicles are examples of where latency is a life-or-death situation. I think everyone would benefit from playing with OpenRTOS, or writing some code that deals with video/audio where there are hard deadlines on latency. But I'm never gonna hold some weekend-project static site generator in Ruby to the same standard as macOS.
  - makapuf 4y ago
    Agreed with what you said, but even for Blender, performance is important but the feature are Free software, good modeler, correct, feature-packed, good looking renderer AND performance.
- agumonkey 4y ago
  aws recently blogged about price benefits of using more performant compiled languages on their servers, it's coming
- Jtsummers 4y ago
  Probably why he had the next paragraph: > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers, it makes sense to think hard about performance at design time. Your scenario falls under that category.
  - anonymoushn 4y ago
    It's the other way around right? If you think about performance, you won't need to orchestrate a fleet.
  - _gabe_ 4y ago
    > Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators This falls into this? > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers A static site generator is a simple utility that should take 100ms at most on a modern crappy laptop. It's not some backend service that's in a critical path, but it's also not a difficult engineering task. Parse input, produce output. This isn't something that should take seconds, which I think is what the OP was getting at. But because of the language choices made, and the millions of lines of bloated code, it does take seconds.
    - Jtsummers 4y ago
      Yes. Maybe not directly, but more like it versus a one-off "I just made a new blog post and can wait 10 seconds for it to render and deploy." If you're reloading the blog posts live then you're on the critical path, not a one-off anymore. So you need to think about performance. In the former case, the performance really doesn't matter. If you've got a personal blog, does it matter if your update takes 10 seconds or 1 second? Probably not, if it does it's the most important blog in the world. If you've got customers with many blogs and need to take their updates and render them, then the performance matters because it's shifted from 1 to many (hundreds? thousands?). And now that 10 second delay is a big issue, you're either using a fleet of servers to handle the load or some customers don't see updates for days (oops).
- paganel 4y ago
  It was the "bad" untyped languages like PHP, Python and Ruby (Perl was too complicated for us, mere mortals) that saved the web from becoming a Microsoft monopoly, or, more likely, an oligopoly between the same MS, probably Sun, probably IBM or some such. I'm talking about most of the 2000s decade. True, the web has become sort of an oligopoly right now, but at least that was not caused by the programming languages and web frameworks that power it. What I'm trying to say is that those languages that everyone is quick to judge right now have given us 10, maybe 15 years of extra "life", a period when most of us have "made" our careers and, well, most of our money (those who have managed to make that money, that is). We wouldn't have had (what basically are) web companies worth tens, if not hundreds of billions of dollars, if the web had still meant relying on Struts or on whatever it is Microsoft was putting forward as a web framework in the mid-2000s. We wouldn't have had engineers taking home TC worth 500-600k and then complaining that Python or Ruby are not what the world needs.
  - rentfree-media 4y ago
    But you might have a viable visual page builder. It's honestly a tough choice at times...
    - j-james 4y ago
      BlueGriffon is pretty good.
  - pdimitar 4y ago
    Interesting claim, any written proof to back it up?
    - paganel 4y ago
      It would be nice to have some University poor souls study that period from a sociological pov (and not only) but, helas, afaik nobody is doing that so that scattered comments like mine, from people who have lived through that period, is the only thing that we have. Come to think of it maybe some data historians at Google can look through the Googlebot logs and not only and compare the percent of websites that they think were .php-run (let’s say) over all this period, in both nominal and relative terms. It is my hunch that those websites represented a big chunk of Googlebot’s hit targets for most of Google’s (the company) existence, hence why I think my comment is valid (among other things)
jbverschoor 4y ago
> CPU time is always cheaper than an engineer’s time. I hate this quote, but less than the "Memory is cheap" mantra.. For "CPU time", if it's a critical path for something with a lot of users and/or where performance is key, the engineer's time is just a fraction.
- bagels 4y ago
  The critical factor is scale. Thousands of servers replaced with hundreds by more efficient code can be worthwhile.
  - TimPC 4y ago
    This means working at companies with massive scale can be far better because you get to make code performant and optimize things rather than just focus on getting features out the door.
    - jbverschoor 4y ago
      Yup. It should, and in general it is.. Look at operating systems, compared to <insert random app>. Somehow programmers care about the big O notation, but not when it's about other people's time.
      - curtainsforus 4y ago
        The operating systems that get fatter and slower every year?
  - djmips 4y ago
    That's not the only factor. Sometimes it's weight or power or a fixed system like embedded, console or a particular product that's not easily upgradable.
  - jcalabro 4y ago
    Yeah for sure. One thing I think about often is if you're writing a compiler and it's slow (say it takes 10s per build), if you have 1000's of engineers running it 100x per day, that starts to add up quick. If you could get it down to 1 second, then you'd save a lot of actual engineer time, just not your own. Scale is key here, but fast software is always a much better user experience than slow stuff as well. With the compiler example, if it takes 1s as opposed to 1h, then users can iterate much more quickly and get a lot of flexibility.
huachimingo 4y ago
Which one is faster? (C code) Return: see if abs(num) > x. /logical comparison/ int greater_abs(int num, int x){ return (num > x) || (num+x < 0); } /squared approach/ int greater_abs2(int num, int x){ return num*num > x; } See it by yourself, with (and without) optimizations: https://godbolt.org/ https://godbolt.org/ What would happen if x is a compile-time constant?
- WalterGR 4y ago
  No idea. How frequently am I calling `abs`?
  - jjice 4y ago
    If you write math heavy code, probably a lot more than if you're writing web apps. Depends on what kind of software you write.
    - WalterGR 4y ago
      Got it. Well if that were the case, I’d use a profiler to see if spending time on optimizing ‘abs’ would realistically be worth it.
  - deleted 4y ago
    [deleted]
- throwaway744678 4y ago
  I don't know which one is faster, but I know that one is not correct (squared approach).
  - saghm 4y ago
    Wouldn't the second one also potentially be incorrect due to overflow?
    - pjscott 4y ago
djmips 4y ago
premature optimization is the root of all evil. "I approve too! But… Sometimes it just has to be fast, and sometimes that means the performance has to be designed in. Nelson agrees and has smart things to say on the subject." - so... you're saying premature optimization isn't the root of all evil. Maybe it's time to retire this tired 'conventional wisdom'
- dralley 4y ago
  Not to mention the paragraph that immediately follows the more famous one, in Djiksta's essay: >> Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.
- dahart 4y ago
  > you're saying premature optimization isn't the root of all evil. Maybe it's time to retire this tired 'conventional wisdom' Funny you mention it. I have a bit of a habit now of reminding people what the remainder of Knuth’s quote actually was. “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” The irony of walking away with the impression that Knuth was saying to not do optimization is that his point was the exact opposite of that, he was emphasizing the word premature, and then saying we absolutely should optimize, after profiling. This all agrees completely with the article’s takeaways: build it right first, then measure, and then optimize the stuff that you can see is the slowest.
  - astrange 4y ago
    The 97% does have its own performance opportunities; rather than making it faster you want to stop it from getting in the way of the important stuff, by reducing code size or tendency to stomp on all the caches or things like that. Anyone optimizing via microbenchmarks or wall time exclusively isn’t going to see this.
- doodpants 4y ago
1vuio0pswjnm7 4y ago
Can anyone state the JSON parsing problem more concisely. For example, 1. Here is the input, e.g., https://data.sfgov.org/api/views/acdm-wktn/rows.json?accessType=DOWNLOAD https://data.sfgov.org/api/views/acdm-wktn/rows.json?accessT... 2. Here is the desired output, e.g., a sample showing what the output is supposed to look like
- pclmulqdq 4y ago
  Having written a few of them, the input is well-defined but the output is not. It generally depends on what you want to do with the JSON parser. If you are trying to get a generic key-value output, you can do that with a hash table, but you have to still access the keys by name. If you want to populate a struct, you should do something different for maximum speed (parse the fields directly).
- dahart 4y ago
  There were several JSON problems mentioned in the article. One was parsing large arrays & floats. Another was DOM style parsing vs streaming. The point of using a streaming JSON parser is to avoid building an "output" per se, and instead handle the elements of the input as though they're events, concurrently with reading the input. In other words, turn parsing into an event handling system with many small events, rather than producing a single large data structure. One reason to do this is to avoid string allocation for every key and string value in the input, because that's where a huge chunk of the time goes when you use a non-streaming parser. Some people call this parsing style SAX parsing (https://en.wikipedia.org/wiki/Simple_API_for_XML https://en.wikipedia.org/wiki/Simple_API_for_XML) Here's an example https://rapidjson.org/md_doc_sax.html https://rapidjson.org/md_doc_sax.html
- 1vuio0pswjnm7 4y ago
  Another way to state the question is, "If the input was in a format that required no JSON processing, nor other processing, what would that format look like?" For example, maybe it is a simple key,value table. I do not know. What makes it easier for me to understand is a sample of what that format looks like.
lumost 4y ago
I'm honestly skeptical if > Make it work, then make it right, then make it fast. Applies to core language choice, I used to hear a lot about rewriting interpreted langs to something faster... but the reality is that a team that's just spent 1 year making an app in python aren't going to pivot to writing Java, Go, or Rust one day. The new team isn't going to be building something new and exciting, they are going to be on a tight timeline to deliver a port of something which already exists.
- secondcoming 4y ago
  Maybe, I mainly use python as a more sophisticated bash script, or to prototype something I'll port to a performant language later.
- edflsafoiewq 4y ago
  IME fast software doesn't actually use this "waterfall" model. There needs to be feedback from performance considerations into semantics.
- welder 4y ago
  Just did this, rewrote a deployed cli from Python[1] to Go[2]. It delivered on all the expectations. Less CPU & RAM usage, less dependence on system stuff like openssl, more diverse platform support. [1] https://github.com/wakatime/legacy-python-cli https://github.com/wakatime/legacy-python-cli [2] https://github.com/wakatime/wakatime-cli https://github.com/wakatime/wakatime-cli
- ChrisMarshallNY 4y ago
  In my case, the choice is made for me (I write Apple native, so Swift, it is...). I have found that his advice on using profilers was very important. I thought I knew where it would be slow, but the profiler usually stuck its tongue out at me and laughed. When I ran a C++ shop, optimization was a black art. Things like keeping code and pipelines in low-level caches could have 100X impact on performance, so we would do things like optimize to avoid breaking cache. This often resulted in things like copying and pasting lines of code to run sequentially, instead of in a loop, or as a subroutine/method. It's difficult. There's usually some "low-hanging fruit" that will give awesome speedups, then, it gets difficult.
runevault 4y ago
The part where he talks about the quality of benchmarking tools reminded me, a library I think worth mentioning for the .NET crowd is BenchmarkdotNet. Not only can it tell you time for a given test to run (after doing warmups to try and mitigate various problems with running tests cold), it also has options to see how the GC performed, at every generation level. Is this code getting to later generations, or is it staying ephemeral which is cheaper to use and destroy? Or can you avoid heap allocations entirely? Edit: Oh and I should mention if you want the method level tracking on a run similar to some of his screenshots, if you pay for a Resharper license that comes with dotTrace I believe which gives you that sort of tracking.
- GordonS 4y ago
  dotTrace fantastic, really essential for performance profiling. Likewise, dotMemory is really good when trying to reduce or understand memory usage (tho dotTrace does have some memory tooling too). I've been happily paying for a JetBrains Ultimate personal license for a few years now. There are very few companies that I'm really rooting for, but JetBrains is absolutely one.
  - runevault 4y ago
    I have a resharper ultimate license through work and a full Jetbrains ultimate at home (I switched to Rider for C#/F#/Unity dev in the past 6 months and really liking it, along with CLion for the times I'm writing rust). One time at work I dug up something that removed 75% of the runtime of an application because it turned out taking the length of an image was actually a method even though it looked like a simple property, so I cached it at the start of processing each image instead of foring over it over and over. It was insane how much faster the code became. I tracked that down with dotTrace. And yeah dotMemory is also fantastic, I've dug up some GNARLY memory usage with it. Probably should have mentioned it since I was bringing up the memory portion of BenchmarkdotNet.
deleted 4y ago
[deleted]
cube2222 4y ago
Go's built-in performance analysis tooling is so excellent. The profiler, which can do CPU, memory, goroutines, blocking, etc. and can display all of that as a graph or a flame graph, as well as `go tool trace` which gives you a full execution trace, including lots of details about the work the GC does. All that with interactive local web-based viewers. Performance optimization is always so fun with it.
- timbray 4y ago
  Yep, but you know, I can't stand that web-based viewer, it's got this hair trigger zoom and if I breathe in the direction of the mouse the graph goes hurtling in or out. I used to look at the PDFs but now I just stay in GoLand, which give you everything you need.
mathiasrw 4y ago
You cant make code faster. You can only make it do less things.
- dr-detroit 4y ago
- hinkley 4y ago
  You can. You can make it do different things, you can make it do the same things in a better order. Most real software has a complicated relationship with Amdahl's Law, and there are often ways to reduce latency without really changing the ops/sec. That's true for IPC, but also true for the processor itself. Sometimes using the 'wrong' instruction for an operation increases the instructions per cycle.
- yjftsjthsd-h 4y ago
  That's either wrong, or right in a way that requires playing with definitions until it's meaningless. Take the traditional example, sorting algorithms; they all do the same thing, but at wildly varying speeds. The only way to claim that some do less is to claim that they use fewer instructions (loosely speaking; pretend that CPU operations are constant time and nothing else matters), at which point you're right but claiming a meaningless tautology. Removing work is a strategy for optimization (sometimes a good one!), but it's not the end-all be-all.
phkahler 4y ago
I'd like to see static analysis that shows the call graph with number of nested loops at each level. Perhaps a way to show the loop lines themselves to make it easy to estimate O(nmk) by looking at the output. Obviously more detailed complexity analysis would involve a person going through the code manually. We have a number of algorithms in Solvespace that are O(n*2) some of them accidentally because certain list operations are O(n). One of the worst offenders was accidentally O(n*4) because an n*2 size input was handed to an O(n*2) algorithm. I wrote a smarter algorithm to avoid that inner call entirely. TLDR it shouldn't be hard for tooling to produce useful time complexity estimates. Can we get this please?
- hinkley 4y ago
  There's a pretty simple workaround for this that I discovered fairly early on: Create a deterministic test for your code, write down the expected call counts for each function, and then look at the actual call counts. You can spend all week shaving 15% off of an inner loop function or you can notice that it's being called 3x as much as you expected, fix that problem, and get a 60% performance improvement without changing a single line of that function.
ntonozzi 4y ago
> The other two languages I’ve used mostly in recent decades are Java and Ruby and the profiler situation is for both those languages is kind of shitty. I had to pay real money to get the Java profiler I used at AWS and while it worked, it was klunky, not fun to use. These days, async profiler (https://github.com/jvm-profiling-tools/async-profiler https://github.com/jvm-profiling-tools/async-profiler) is much better than the Go tooling for performance. It is a joy to use and features a top-like view for the hottest methods. It works for locks, allocations and CPU time. It also integrates with JMH.
hinkley 4y ago
> CPU time is always cheaper than an engineer’s time. But my time is sometimes cheaper than 10 other developer's time. Or 50. Or all of our customers now and in the future: https://xkcd.com/1205/ https://xkcd.com/1205/ Additionally, a fatal flaw of most project management theory is that it measures time as the constrained resource, which is rarely if ever the case. The real constraint is not time. It's either attention or energy, depending on the context (and motivation). We are not machines. Taking 10 minutes off of a process doesn't necessarily allow us to get 10 more minutes of work done, while taking 5 minutes off of another may actually save us 15.
- john-tells-all 4y ago
  Absolutely. As an Engineer, I've found the Pomodoro technique to be invaluable. I work with high focus for a 20-minute "cycle", get up and walk around, then repeat. No faffing about while working, and the mild exercise stimulates the brain so my "on time" is very effective.
hayley-patton 4y ago
> I had to pay real money to get the Java profiler I used at AWS and while it worked, it was klunky, not fun to use. VisualVM <https://visualvm.github.io/ https://visualvm.github.io/> is quite good for Java/JVM profiling.
neonsunset 4y ago
Premature optimization has become an umbrella term for people defending poor choices during implementation. Not doing simple mistakes that kill performance for no benefit or choosing option B over A where both take equal effort but B is 20x times more performant should be common sense. Unfortunately, 9 times out of 10 I hear these often quoted phrases is when engineers defend their status quo bias during code review.
- infogulch 4y ago
  I like Casey Miratory's "non-pessimization" approach in Philosophies of Optimization [1] [2]. He breaks "making a program go faster" into 3 modes: (1) "optimization" which includes counting expected cycles or memory bandwidth, careful benchmarking, designing custom algorithms, etc which can be fun but is expensive and is only rarely applied; (2) "non-pessimization" which is simply avoiding adding extra unnecessary work for the cpu, which should be done 100% of the time but isn't in the industry today; (3) and "fake optimization" which is shallowly repeating 'optimization tips' and conclusions from unrelated cases of (1) without actually measuring it. [1]: https://www.youtube.com/watch?v=pgoetgxecw8 https://www.youtube.com/watch?v=pgoetgxecw8 [2]: https://www.youtube.com/watch?v=lStYLF6Us_Q https://www.youtube.com/watch?v=lStYLF6Us_Q
  - neonsunset 4y ago
    Thank you for the links, looks like something definitely worth checking out, especially the name of the whole series "Simple Code, High Performance" because it sounds like the core of my approach to writing code. "non-pessimization" is my most loved technique because a design reduced to the minimum amount of necessary work and abstractions is more often than not has better performance and is easier to reason about.
    - infogulch 4y ago
      The whole series is pretty great. One of the interesting points he makes is that non-pessimization (2) is a necessary precondition to optimization (1). That any attempt to do hardcore optimization of a pessimized codebase is futile and will continuously faceplant into endless expanses of stupid code filled with unnecessary work for the cpu that instantly becomes the bottleneck.

149 points by zdw 4y ago

> If you’re writing Ruby code to turn your blog drafts into HTML, it doesn’t much matter if republishing takes a few extra seconds Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators and it takes ~5,000 ms on an i9 machine when it could take 100 ms if different languages and different implementation choices were made. This is the story of modern software: "I don't care how much time and computing power I waste. It's not the core app of some FAANG giant, so I'm not going to bother, ever."

bluedino 4y ago
It's times like these when we forget Turbo Pascal compiling 40,000 lines per second on a 386
dahfizz 4y ago
I wonder how heavily other fields sacrifice in the name of "ease of implementation". Could we have houses that are 100x stronger and longer lasting if we allowed a few extra weeks of construction time? Could we 10x battery capacity with a slightly more sophisticated manufacturing process? I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software.
- ajmurmann 4y ago
  For the economics of that comparison to make sense, we should also include material inputs to the construction. It doesn't matter to the cost of the final product if it was materials that were saved or labor cost. Software is the special case where cost almost entirely comes down to labor. So given that, we are of course constantly compromising quality. We realize that less though because buildings are more standardized.
- TimPC 4y ago
  The big issue in other fields mostly isn't manufacturing design or construction process. The main bottleneck is with physical goods and long distances travelled you're subject to high shipping costs so people cut all sorts of corners to make things less bulky and lighter. Cheap plastics are far lighter than wood or metals for instance so we see more plastics get used.
- adamdusty 4y ago
  I work in III/V semiconductors. The product development goes like: identify need, design, attempt a proof of concept, DoEs to determine manufacturing costs/viability, repeat until you have a statistically controlled viable process. There is essentially no room for technical creativity like there is in software. If we spent an extra year on our worst performing product (we've done this), we would get at best 5% improvement with iffy reproducibility. I dont know about construction or batteries.
- astrange 4y ago
  You could have a lot more good enough and much cheaper houses in the US, but we banned manufactured/prefab houses due to lobbying from the “build everything individually out of wood” lobby, require huge setbacks due to the front lawn lobby, and various other things like overly wide roads because of out of date fire codes.
- mr_toad 4y ago
  > I don't think many developers nowadays understand how fast computers are, nor how much needless bloat is in their software. Most, as in > 50%, of developers aren’t in a position to decide which languages, libraries and tools to use, or how much time they can spend on performance improvement.
- Gigachad 4y ago
  Anyone with APM setup can see what’s going on pretty well. For almost all web apps, you can see that ruby/python are extremely fast and almost all of the time spent on a request is the database.
  - dgb23 4y ago
    The implication is likely that the db does all the work, and the parts in ruby/python have barely anything to do.
    - Gigachad 4y ago
      This is true of every rails app I have worked on in the last 10 years. You’d have to be doing something remarkably bad or have some crazy situation to end up with ruby using any noticeable amount of compute time.
      - dgb23 4y ago
        Well that depends on what your application is doing. But yes, if it is a database heavy workload, then what you say applies. What I was trying to get at was that in these cases the ruby/python/whatever parts are not actually fast, they just do much less stuff. Add that the database might be on another machine, so there's plenty of overhead as well. Aside: You can sometimes get orders of magnitude of speed gains just by using an embedded database and a fast language on top if your workload allows it. Essentially you don't really do much to get this, just cut overhead.
dwrodri 4y ago
I agree to the extent that people fail to realize how they could be missing out opportunities to innovate or corner a market when they leave performance on the table. Quite often, new products become possible when a basic task like rendering HTML goes from taking 10 seconds to 10ms. I think you can paint the problem in broad strokes from either side of the "how important is performance?" argument. From a "get the bills paid" point-of-view, any good project manager also has to know when to tell an engineer to focus on getting the product shipped instead of chasing that next 5% in throughput/latency reduction/overhead. I've seen my fair share of programmers (including myself) refuse to ship to a project because the pursuit of some "undesirable" latency and not finish more important features. For tasks like video streaming, Automation software (CI pipelines to robotics), video games, professional tools for content creation (DAWs, video editing, Blender, etc.) performance is the feature, but then your product is helping them get the bills paid faster. Medical apparatus(es?) and guidance software on autonomous vehicles are examples of where latency is a life-or-death situation. I think everyone would benefit from playing with OpenRTOS, or writing some code that deals with video/audio where there are hard deadlines on latency. But I'm never gonna hold some weekend-project static site generator in Ruby to the same standard as macOS.
- makapuf 4y ago
  Agreed with what you said, but even for Blender, performance is important but the feature are Free software, good modeler, correct, feature-packed, good looking renderer AND performance.
agumonkey 4y ago
aws recently blogged about price benefits of using more performant compiled languages on their servers, it's coming
Jtsummers 4y ago
Probably why he had the next paragraph: > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers, it makes sense to think hard about performance at design time. Your scenario falls under that category.
- anonymoushn 4y ago
  It's the other way around right? If you think about performance, you won't need to orchestrate a fleet.
- _gabe_ 4y ago
  > Unless your software is intended to reload blog posts live like Hugo/Jekyll/other static site generators This falls into this? > But for code that’s on the critical path of a service back-end running on a big sharded fleet with lots of zeroes in the “per second” numbers A static site generator is a simple utility that should take 100ms at most on a modern crappy laptop. It's not some backend service that's in a critical path, but it's also not a difficult engineering task. Parse input, produce output. This isn't something that should take seconds, which I think is what the OP was getting at. But because of the language choices made, and the millions of lines of bloated code, it does take seconds.
  - Jtsummers 4y ago
    Yes. Maybe not directly, but more like it versus a one-off "I just made a new blog post and can wait 10 seconds for it to render and deploy." If you're reloading the blog posts live then you're on the critical path, not a one-off anymore. So you need to think about performance. In the former case, the performance really doesn't matter. If you've got a personal blog, does it matter if your update takes 10 seconds or 1 second? Probably not, if it does it's the most important blog in the world. If you've got customers with many blogs and need to take their updates and render them, then the performance matters because it's shifted from 1 to many (hundreds? thousands?). And now that 10 second delay is a big issue, you're either using a fleet of servers to handle the load or some customers don't see updates for days (oops).
paganel 4y ago
It was the "bad" untyped languages like PHP, Python and Ruby (Perl was too complicated for us, mere mortals) that saved the web from becoming a Microsoft monopoly, or, more likely, an oligopoly between the same MS, probably Sun, probably IBM or some such. I'm talking about most of the 2000s decade. True, the web has become sort of an oligopoly right now, but at least that was not caused by the programming languages and web frameworks that power it. What I'm trying to say is that those languages that everyone is quick to judge right now have given us 10, maybe 15 years of extra "life", a period when most of us have "made" our careers and, well, most of our money (those who have managed to make that money, that is). We wouldn't have had (what basically are) web companies worth tens, if not hundreds of billions of dollars, if the web had still meant relying on Struts or on whatever it is Microsoft was putting forward as a web framework in the mid-2000s. We wouldn't have had engineers taking home TC worth 500-600k and then complaining that Python or Ruby are not what the world needs.
- rentfree-media 4y ago
  But you might have a viable visual page builder. It's honestly a tough choice at times...
  - j-james 4y ago
    BlueGriffon is pretty good.
- pdimitar 4y ago
  Interesting claim, any written proof to back it up?
  - paganel 4y ago
    It would be nice to have some University poor souls study that period from a sociological pov (and not only) but, helas, afaik nobody is doing that so that scattered comments like mine, from people who have lived through that period, is the only thing that we have. Come to think of it maybe some data historians at Google can look through the Googlebot logs and not only and compare the percent of websites that they think were .php-run (let’s say) over all this period, in both nominal and relative terms. It is my hunch that those websites represented a big chunk of Googlebot’s hit targets for most of Google’s (the company) existence, hence why I think my comment is valid (among other things)

jbverschoor 4y ago

> CPU time is always cheaper than an engineer’s time. I hate this quote, but less than the "Memory is cheap" mantra.. For "CPU time", if it's a critical path for something with a lot of users and/or where performance is key, the engineer's time is just a fraction.

bagels 4y ago
The critical factor is scale. Thousands of servers replaced with hundreds by more efficient code can be worthwhile.
- TimPC 4y ago
  This means working at companies with massive scale can be far better because you get to make code performant and optimize things rather than just focus on getting features out the door.
  - jbverschoor 4y ago
    Yup. It should, and in general it is.. Look at operating systems, compared to <insert random app>. Somehow programmers care about the big O notation, but not when it's about other people's time.
    - curtainsforus 4y ago
      The operating systems that get fatter and slower every year?
- djmips 4y ago
  That's not the only factor. Sometimes it's weight or power or a fixed system like embedded, console or a particular product that's not easily upgradable.
- jcalabro 4y ago
  Yeah for sure. One thing I think about often is if you're writing a compiler and it's slow (say it takes 10s per build), if you have 1000's of engineers running it 100x per day, that starts to add up quick. If you could get it down to 1 second, then you'd save a lot of actual engineer time, just not your own. Scale is key here, but fast software is always a much better user experience than slow stuff as well. With the compiler example, if it takes 1s as opposed to 1h, then users can iterate much more quickly and get a lot of flexibility.

huachimingo 4y ago

Which one is faster? (C code) Return: see if abs(num) > x. /logical comparison/ int greater_abs(int num, int x){ return (num > x) || (num+x < 0); } /squared approach/ int greater_abs2(int num, int x){ return num*num > x; } See it by yourself, with (and without) optimizations: https://godbolt.org/ https://godbolt.org/ What would happen if x is a compile-time constant?

WalterGR 4y ago
No idea. How frequently am I calling `abs`?
- jjice 4y ago
  If you write math heavy code, probably a lot more than if you're writing web apps. Depends on what kind of software you write.
  - WalterGR 4y ago
    Got it. Well if that were the case, I’d use a profiler to see if spending time on optimizing ‘abs’ would realistically be worth it.
- deleted 4y ago
  [deleted]
throwaway744678 4y ago
I don't know which one is faster, but I know that one is not correct (squared approach).
- saghm 4y ago
  Wouldn't the second one also potentially be incorrect due to overflow?
  - pjscott 4y ago

djmips 4y ago

premature optimization is the root of all evil. "I approve too! But… Sometimes it just has to be fast, and sometimes that means the performance has to be designed in. Nelson agrees and has smart things to say on the subject." - so... you're saying premature optimization isn't the root of all evil. Maybe it's time to retire this tired 'conventional wisdom'

dralley 4y ago
Not to mention the paragraph that immediately follows the more famous one, in Djiksta's essay: >> Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.
dahart 4y ago
> you're saying premature optimization isn't the root of all evil. Maybe it's time to retire this tired 'conventional wisdom' Funny you mention it. I have a bit of a habit now of reminding people what the remainder of Knuth’s quote actually was. “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.” The irony of walking away with the impression that Knuth was saying to not do optimization is that his point was the exact opposite of that, he was emphasizing the word premature, and then saying we absolutely should optimize, after profiling. This all agrees completely with the article’s takeaways: build it right first, then measure, and then optimize the stuff that you can see is the slowest.
- astrange 4y ago
  The 97% does have its own performance opportunities; rather than making it faster you want to stop it from getting in the way of the important stuff, by reducing code size or tendency to stomp on all the caches or things like that. Anyone optimizing via microbenchmarks or wall time exclusively isn’t going to see this.
doodpants 4y ago

1vuio0pswjnm7 4y ago

Can anyone state the JSON parsing problem more concisely. For example, 1. Here is the input, e.g., https://data.sfgov.org/api/views/acdm-wktn/rows.json?accessType=DOWNLOAD https://data.sfgov.org/api/views/acdm-wktn/rows.json?accessT... 2. Here is the desired output, e.g., a sample showing what the output is supposed to look like

pclmulqdq 4y ago
Having written a few of them, the input is well-defined but the output is not. It generally depends on what you want to do with the JSON parser. If you are trying to get a generic key-value output, you can do that with a hash table, but you have to still access the keys by name. If you want to populate a struct, you should do something different for maximum speed (parse the fields directly).
dahart 4y ago
There were several JSON problems mentioned in the article. One was parsing large arrays & floats. Another was DOM style parsing vs streaming. The point of using a streaming JSON parser is to avoid building an "output" per se, and instead handle the elements of the input as though they're events, concurrently with reading the input. In other words, turn parsing into an event handling system with many small events, rather than producing a single large data structure. One reason to do this is to avoid string allocation for every key and string value in the input, because that's where a huge chunk of the time goes when you use a non-streaming parser. Some people call this parsing style SAX parsing (https://en.wikipedia.org/wiki/Simple_API_for_XML https://en.wikipedia.org/wiki/Simple_API_for_XML) Here's an example https://rapidjson.org/md_doc_sax.html https://rapidjson.org/md_doc_sax.html
1vuio0pswjnm7 4y ago
Another way to state the question is, "If the input was in a format that required no JSON processing, nor other processing, what would that format look like?" For example, maybe it is a simple key,value table. I do not know. What makes it easier for me to understand is a sample of what that format looks like.

lumost 4y ago

I'm honestly skeptical if > Make it work, then make it right, then make it fast. Applies to core language choice, I used to hear a lot about rewriting interpreted langs to something faster... but the reality is that a team that's just spent 1 year making an app in python aren't going to pivot to writing Java, Go, or Rust one day. The new team isn't going to be building something new and exciting, they are going to be on a tight timeline to deliver a port of something which already exists.

secondcoming 4y ago
Maybe, I mainly use python as a more sophisticated bash script, or to prototype something I'll port to a performant language later.
edflsafoiewq 4y ago
IME fast software doesn't actually use this "waterfall" model. There needs to be feedback from performance considerations into semantics.
welder 4y ago
Just did this, rewrote a deployed cli from Python[1] to Go[2]. It delivered on all the expectations. Less CPU & RAM usage, less dependence on system stuff like openssl, more diverse platform support. [1] https://github.com/wakatime/legacy-python-cli https://github.com/wakatime/legacy-python-cli [2] https://github.com/wakatime/wakatime-cli https://github.com/wakatime/wakatime-cli
ChrisMarshallNY 4y ago
In my case, the choice is made for me (I write Apple native, so Swift, it is...). I have found that his advice on using profilers was very important. I thought I knew where it would be slow, but the profiler usually stuck its tongue out at me and laughed. When I ran a C++ shop, optimization was a black art. Things like keeping code and pipelines in low-level caches could have 100X impact on performance, so we would do things like optimize to avoid breaking cache. This often resulted in things like copying and pasting lines of code to run sequentially, instead of in a loop, or as a subroutine/method. It's difficult. There's usually some "low-hanging fruit" that will give awesome speedups, then, it gets difficult.

runevault 4y ago

The part where he talks about the quality of benchmarking tools reminded me, a library I think worth mentioning for the .NET crowd is BenchmarkdotNet. Not only can it tell you time for a given test to run (after doing warmups to try and mitigate various problems with running tests cold), it also has options to see how the GC performed, at every generation level. Is this code getting to later generations, or is it staying ephemeral which is cheaper to use and destroy? Or can you avoid heap allocations entirely? Edit: Oh and I should mention if you want the method level tracking on a run similar to some of his screenshots, if you pay for a Resharper license that comes with dotTrace I believe which gives you that sort of tracking.

GordonS 4y ago
dotTrace fantastic, really essential for performance profiling. Likewise, dotMemory is really good when trying to reduce or understand memory usage (tho dotTrace does have some memory tooling too). I've been happily paying for a JetBrains Ultimate personal license for a few years now. There are very few companies that I'm really rooting for, but JetBrains is absolutely one.
- runevault 4y ago
  I have a resharper ultimate license through work and a full Jetbrains ultimate at home (I switched to Rider for C#/F#/Unity dev in the past 6 months and really liking it, along with CLion for the times I'm writing rust). One time at work I dug up something that removed 75% of the runtime of an application because it turned out taking the length of an image was actually a method even though it looked like a simple property, so I cached it at the start of processing each image instead of foring over it over and over. It was insane how much faster the code became. I tracked that down with dotTrace. And yeah dotMemory is also fantastic, I've dug up some GNARLY memory usage with it. Probably should have mentioned it since I was bringing up the memory portion of BenchmarkdotNet.

deleted 4y ago

[deleted]

cube2222 4y ago

Go's built-in performance analysis tooling is so excellent. The profiler, which can do CPU, memory, goroutines, blocking, etc. and can display all of that as a graph or a flame graph, as well as `go tool trace` which gives you a full execution trace, including lots of details about the work the GC does. All that with interactive local web-based viewers. Performance optimization is always so fun with it.

timbray 4y ago
Yep, but you know, I can't stand that web-based viewer, it's got this hair trigger zoom and if I breathe in the direction of the mouse the graph goes hurtling in or out. I used to look at the PDFs but now I just stay in GoLand, which give you everything you need.

mathiasrw 4y ago

You cant make code faster. You can only make it do less things.

dr-detroit 4y ago
hinkley 4y ago
You can. You can make it do different things, you can make it do the same things in a better order. Most real software has a complicated relationship with Amdahl's Law, and there are often ways to reduce latency without really changing the ops/sec. That's true for IPC, but also true for the processor itself. Sometimes using the 'wrong' instruction for an operation increases the instructions per cycle.
yjftsjthsd-h 4y ago
That's either wrong, or right in a way that requires playing with definitions until it's meaningless. Take the traditional example, sorting algorithms; they all do the same thing, but at wildly varying speeds. The only way to claim that some do less is to claim that they use fewer instructions (loosely speaking; pretend that CPU operations are constant time and nothing else matters), at which point you're right but claiming a meaningless tautology. Removing work is a strategy for optimization (sometimes a good one!), but it's not the end-all be-all.

phkahler 4y ago

I'd like to see static analysis that shows the call graph with number of nested loops at each level. Perhaps a way to show the loop lines themselves to make it easy to estimate O(nmk) by looking at the output. Obviously more detailed complexity analysis would involve a person going through the code manually. We have a number of algorithms in Solvespace that are O(n*2) some of them accidentally because certain list operations are O(n). One of the worst offenders was accidentally O(n*4) because an n*2 size input was handed to an O(n*2) algorithm. I wrote a smarter algorithm to avoid that inner call entirely. TLDR it shouldn't be hard for tooling to produce useful time complexity estimates. Can we get this please?

hinkley 4y ago
There's a pretty simple workaround for this that I discovered fairly early on: Create a deterministic test for your code, write down the expected call counts for each function, and then look at the actual call counts. You can spend all week shaving 15% off of an inner loop function or you can notice that it's being called 3x as much as you expected, fix that problem, and get a 60% performance improvement without changing a single line of that function.

ntonozzi 4y ago

> The other two languages I’ve used mostly in recent decades are Java and Ruby and the profiler situation is for both those languages is kind of shitty. I had to pay real money to get the Java profiler I used at AWS and while it worked, it was klunky, not fun to use. These days, async profiler (https://github.com/jvm-profiling-tools/async-profiler https://github.com/jvm-profiling-tools/async-profiler) is much better than the Go tooling for performance. It is a joy to use and features a top-like view for the hottest methods. It works for locks, allocations and CPU time. It also integrates with JMH.

hinkley 4y ago

> CPU time is always cheaper than an engineer’s time. But my time is sometimes cheaper than 10 other developer's time. Or 50. Or all of our customers now and in the future: https://xkcd.com/1205/ https://xkcd.com/1205/ Additionally, a fatal flaw of most project management theory is that it measures time as the constrained resource, which is rarely if ever the case. The real constraint is not time. It's either attention or energy, depending on the context (and motivation). We are not machines. Taking 10 minutes off of a process doesn't necessarily allow us to get 10 more minutes of work done, while taking 5 minutes off of another may actually save us 15.

john-tells-all 4y ago
Absolutely. As an Engineer, I've found the Pomodoro technique to be invaluable. I work with high focus for a 20-minute "cycle", get up and walk around, then repeat. No faffing about while working, and the mild exercise stimulates the brain so my "on time" is very effective.

hayley-patton 4y ago

> I had to pay real money to get the Java profiler I used at AWS and while it worked, it was klunky, not fun to use. VisualVM <https://visualvm.github.io/ https://visualvm.github.io/> is quite good for Java/JVM profiling.

neonsunset 4y ago

Premature optimization has become an umbrella term for people defending poor choices during implementation. Not doing simple mistakes that kill performance for no benefit or choosing option B over A where both take equal effort but B is 20x times more performant should be common sense. Unfortunately, 9 times out of 10 I hear these often quoted phrases is when engineers defend their status quo bias during code review.

infogulch 4y ago
I like Casey Miratory's "non-pessimization" approach in Philosophies of Optimization [1] [2]. He breaks "making a program go faster" into 3 modes: (1) "optimization" which includes counting expected cycles or memory bandwidth, careful benchmarking, designing custom algorithms, etc which can be fun but is expensive and is only rarely applied; (2) "non-pessimization" which is simply avoiding adding extra unnecessary work for the cpu, which should be done 100% of the time but isn't in the industry today; (3) and "fake optimization" which is shallowly repeating 'optimization tips' and conclusions from unrelated cases of (1) without actually measuring it. [1]: https://www.youtube.com/watch?v=pgoetgxecw8 https://www.youtube.com/watch?v=pgoetgxecw8 [2]: https://www.youtube.com/watch?v=lStYLF6Us_Q https://www.youtube.com/watch?v=lStYLF6Us_Q
- neonsunset 4y ago
  Thank you for the links, looks like something definitely worth checking out, especially the name of the whole series "Simple Code, High Performance" because it sounds like the core of my approach to writing code. "non-pessimization" is my most loved technique because a design reduced to the minimum amount of necessary work and abstractions is more often than not has better performance and is easier to reason about.
  - infogulch 4y ago
    The whole series is pretty great. One of the interesting points he makes is that non-pessimization (2) is a necessary precondition to optimization (1). That any attempt to do hardcore optimization of a pessimized codebase is futile and will continuously faceplant into endless expanses of stupid code filled with unnecessary work for the cpu that instantly becomes the bottleneck.