6 ms·

Consider: if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning? This is something I've c

Consider: if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning? This is something I've considered a bit since listening to some Jonathan Blow talks. When you have loose requirements which change over time, limited time, and you're left figuring stuff out on your own it's difficult to do an outstanding job. Nobody taught me how to write fast software. At best I learned enough to get stuff working. Usually after I've built something slow but functional I might've learned enough to write a faster version, but there's no time to do another rewrite because the features are changing. For many developers the choice isn't between fast and slow, it's probably between slow or nothing at all. Although I do think that big tech companies should be held to slightly higher standards.

armchairhacker 4y ago
The #1 rule of optimization is that it's only 10% or 1% of the code which matters. You only need to optimize the code which runs 1000 times a second, the code which runs 1 time every 10 seconds in a background thread can be inefficient and nobody will notice or care. You don't need to rewrite your entire app to make it run substantially faster, you only need to rewrite the hot-paths. Another key rule is to limit the "main loop" to as little as possible: don't run big computations which don't affect what the user is focusing on. For example, web browsers will pause and unload tabs when you have several of them: this means you can have over 100 open tabs and your web browser will still run fast, because the unopened tabs are not doing anything, they are just caching the URL and (in some cases) whatever is rendered on the site. Similarly, most games try not to render or update things outside of the player's vision. You don't have to go to extremes like this, but if something doesn't affect the user's flow (e.g. an extra feature only a few people use), its performance impact should be negligible. Specifically: don't put code in your hot-path that doesn't need to be there. Another key rule is to use well-written libraries for your algorithms. I assure you a vector-math library with 1000 stars implements vector-math operations much faster than you can, and those operations are neatly wrapped in easy-to-use functions. When you have code in your hot-path which needs to be there, is too slow and you can't replace it with a library, then you bring out the big-O and zero-allocation techniques. And also, caching. A lot of optimization is ultimately caching, as computation is generally much more expensive than memory. Those super pretty render engines all use a ton of caching. You can have a piece of software with tons of bloat, extra features, inefficient/redundant computations, and an Electron back-end, which is still fast (example: VSCode). Even the Linux kernel has a lot of bloat in the form of various drivers, but it does not affect the main runtime as these drivers are not loaded. Even graphically-intensive games and simulations have redundant computations and excess allocations, they are just not in the hot-path. And last tip: don't write exponential-time algorithms. The above doesn't apply when you have an exponential-time algorithm, because even when n = 30 it will slow your computer, and when n = 75 it will run past the sun burns out.
- JonChesterfield 4y ago
  The optimise the hot part strategy, applied repeatedly across different test cases, stamps down the spikes on the profiler until the profile is basically flat. Sadly at that point rather more thought is needed and it seems to get a bit domain specific what needs to change next. A corollary of this that bad architecture choices made up front, e.g. because performance was not considered, become both extremely entrenched and difficult to see on the flat flame graph.
projektfu 4y ago
First, learn how to run an execution profile[1], like the author did in this article. Second, try to find out how fast software is written in the domain. These are often architectural decisions. Is this software better written by loading the entire data into memory? Or should I process it sequentially and keep memory usage bounded? What data needs to be kept "hot"? Third, try to figure out what parts of the OS or libraries you use have potentially unbounded waiting. You don't want that to be in any thread that has to be responsive. Either make it async or run it in another thread. This is the cause of a lot of slowness in software, where it gets tested mainly on a single machine but it is actually a distributed system, or where there is never contention for a file in testing, nor is a database row ever locked by another application. 1. https://learn.microsoft.com/en-us/visualstudio/profiling/profiling-feature-tour?view=vs-2022#analyze-cpu-usage https://learn.microsoft.com/en-us/visualstudio/profiling/pro...
userbinator 4y ago
but I don't know enough to make it fast I'd say that in the vast majority of cases, you don't actually need to; what you need to instead focus on is how to make it not slow, and to do so is really not that difficult. In a word, it's (total) simplicity. The less code there is in total that the CPU has to execute, the faster it'll be. Thus, don't add unnecessary abstractions/indirections unless necessary, pull in huge libraries and use a tiny fraction of their functionality, etc. I'm of the opinion that you don't need to know about cycle counting and such to write decently efficient software. You don't need to specifically optimise if you start off with a sane and simple design.
- culi 4y ago
  > The less code there is in total that the CPU has to execute, the faster it'll be. The more higher-level the language is, the less this is true in my experience
- Too 4y ago
  Simplicity is good but often not the solution to make things faster. Most obvious example. Binary search is faster than linear, at the cost of more complexity (that can often be hidden under an abstraction). Async code is often perceived as faster than synchronous, at the cost of more complexity. In the case of TFA, loading the file list asynchronously in the background could have been a good candidate for such.
- laundermaf 4y ago
  > focus on is how to make it not slow That's the same thing, worded differently. A junior programmer might focus on looping for-loops backwards "for speed" and then leave a serial `await` in the loop itself. If you don't see that that's a problem, no "focus" will help.
CamelRocketFish 4y ago
Under Big O and any time you have an algorithm, see what the complexity is and if you can reduce it.
Jtsummers 4y ago
Study basic algorithms and data structures. The purpose of this is not to pass a leetcode test, those are obnoxious and those companies that conduct them should be embarrassed. The purpose is to learn how to analyze systems and their performance on two primary metrics: time and space. This isn't all there is to understanding larger systems (simplistic counting the steps of a loop is not sufficient if you're swamped by IO access times, for instance, but it's a start). But you need to be able to examine a system and understand tradeoffs. When deciding between different data structures you're often faced with a situation like: X is faster at A, but slower at B; Y is the opposite. Which do you choose? I can't tell you, you have to know your intended use-cases. Will A dominate in your application, or B? This is the kind of thinking that scales, too. A database that's better at random access reads and writes is better for some applications than one that's optimized for sequential reads but lousy at random writes, so choose the suitable option based on a reasonable expectation of your system's needs. Study parallel programming (that is, actually having two or more threads/processes/whatevers running simultaneously). This will help you understand the limits of parallelism in speeding up programs, but also the potential for some programs. You'll learn about how to divide work across N processors, the limits of performance gains, and where you will get performance losses (critical ones: synchronization and communication). Study concurrency so you can see how to make use of asynchronous patterns. I don't mean strictly async/await style, just how to use whatever language/OS facilities to avoid unnecessary blocking. Like, if you're making a network query it can take seconds or longer to resolve (depending on location, size of the data, remote server load, etc.). Your program shouldn't wait if it still has other things it can do (if it doesn't, then let it wait). Use a profiler, study your real systems and see where they spend their time. What functions get called the most, what functions or syscalls consume the most time. Is there a file/resource that's constantly being used? I once sped up a program by around 60x just by realizing that the previous coder was repeatedly reading from the same file. It wasn't needed, he could have read from it once (optimal), or at least cached it in memory (though memory wasn't so plentiful then, it would've been a tight fit on a mid-00s desktop, but fine today). Apply the ideas above to analyzing it and improving it.
- salawat 4y ago
  On parallelism: Sufficient attentiin applied to parallelism/concurrency, is indistinguishable from management, without any of the benefits or recognition, or most other practitioners bothering to even try to to grok it. Does that mean you shouldn't do it? No. However, there comes a point where if you breach that barrier, you will discover that the fundamental problems of today arise out of willingness to inflict pain stemming from the component actors in the Human Virtual Machine.
blt 4y ago
> if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning? Learn how to use a sampling profiler.
RedShift1 4y ago
Use an old computer. Slow operations are enlarged which makes it more obvious where the problem lies. I have an old HP EliteBook 6930p with a Core 2 Duo P8700 CPU from 2008. I did put an SSD in it because Windows 10 is unusable without it.
- Narishma 4y ago
  Using an SSD hides a lot of IO-related performance issues though. I suspect it's why Windows 10 is super slow on an HDD.

armchairhacker 4y ago

The #1 rule of optimization is that it's only 10% or 1% of the code which matters. You only need to optimize the code which runs 1000 times a second, the code which runs 1 time every 10 seconds in a background thread can be inefficient and nobody will notice or care. You don't need to rewrite your entire app to make it run substantially faster, you only need to rewrite the hot-paths. Another key rule is to limit the "main loop" to as little as possible: don't run big computations which don't affect what the user is focusing on. For example, web browsers will pause and unload tabs when you have several of them: this means you can have over 100 open tabs and your web browser will still run fast, because the unopened tabs are not doing anything, they are just caching the URL and (in some cases) whatever is rendered on the site. Similarly, most games try not to render or update things outside of the player's vision. You don't have to go to extremes like this, but if something doesn't affect the user's flow (e.g. an extra feature only a few people use), its performance impact should be negligible. Specifically: don't put code in your hot-path that doesn't need to be there. Another key rule is to use well-written libraries for your algorithms. I assure you a vector-math library with 1000 stars implements vector-math operations much faster than you can, and those operations are neatly wrapped in easy-to-use functions. When you have code in your hot-path which needs to be there, is too slow and you can't replace it with a library, then you bring out the big-O and zero-allocation techniques. And also, caching. A lot of optimization is ultimately caching, as computation is generally much more expensive than memory. Those super pretty render engines all use a ton of caching. You can have a piece of software with tons of bloat, extra features, inefficient/redundant computations, and an Electron back-end, which is still fast (example: VSCode). Even the Linux kernel has a lot of bloat in the form of various drivers, but it does not affect the main runtime as these drivers are not loaded. Even graphically-intensive games and simulations have redundant computations and excess allocations, they are just not in the hot-path. And last tip: don't write exponential-time algorithms. The above doesn't apply when you have an exponential-time algorithm, because even when n = 30 it will slow your computer, and when n = 75 it will run past the sun burns out.

JonChesterfield 4y ago
The optimise the hot part strategy, applied repeatedly across different test cases, stamps down the spikes on the profiler until the profile is basically flat. Sadly at that point rather more thought is needed and it seems to get a bit domain specific what needs to change next. A corollary of this that bad architecture choices made up front, e.g. because performance was not considered, become both extremely entrenched and difficult to see on the flat flame graph.

projektfu 4y ago

First, learn how to run an execution profile[1], like the author did in this article. Second, try to find out how fast software is written in the domain. These are often architectural decisions. Is this software better written by loading the entire data into memory? Or should I process it sequentially and keep memory usage bounded? What data needs to be kept "hot"? Third, try to figure out what parts of the OS or libraries you use have potentially unbounded waiting. You don't want that to be in any thread that has to be responsive. Either make it async or run it in another thread. This is the cause of a lot of slowness in software, where it gets tested mainly on a single machine but it is actually a distributed system, or where there is never contention for a file in testing, nor is a database row ever locked by another application. 1. https://learn.microsoft.com/en-us/visualstudio/profiling/profiling-feature-tour?view=vs-2022#analyze-cpu-usage https://learn.microsoft.com/en-us/visualstudio/profiling/pro...

userbinator 4y ago

but I don't know enough to make it fast I'd say that in the vast majority of cases, you don't actually need to; what you need to instead focus on is how to make it not slow, and to do so is really not that difficult. In a word, it's (total) simplicity. The less code there is in total that the CPU has to execute, the faster it'll be. Thus, don't add unnecessary abstractions/indirections unless necessary, pull in huge libraries and use a tiny fraction of their functionality, etc. I'm of the opinion that you don't need to know about cycle counting and such to write decently efficient software. You don't need to specifically optimise if you start off with a sane and simple design.

culi 4y ago
> The less code there is in total that the CPU has to execute, the faster it'll be. The more higher-level the language is, the less this is true in my experience
Too 4y ago
Simplicity is good but often not the solution to make things faster. Most obvious example. Binary search is faster than linear, at the cost of more complexity (that can often be hidden under an abstraction). Async code is often perceived as faster than synchronous, at the cost of more complexity. In the case of TFA, loading the file list asynchronously in the background could have been a good candidate for such.
laundermaf 4y ago
> focus on is how to make it not slow That's the same thing, worded differently. A junior programmer might focus on looping for-loops backwards "for speed" and then leave a serial `await` in the loop itself. If you don't see that that's a problem, no "focus" will help.

CamelRocketFish 4y ago

Under Big O and any time you have an algorithm, see what the complexity is and if you can reduce it.

Jtsummers 4y ago

Study basic algorithms and data structures. The purpose of this is not to pass a leetcode test, those are obnoxious and those companies that conduct them should be embarrassed. The purpose is to learn how to analyze systems and their performance on two primary metrics: time and space. This isn't all there is to understanding larger systems (simplistic counting the steps of a loop is not sufficient if you're swamped by IO access times, for instance, but it's a start). But you need to be able to examine a system and understand tradeoffs. When deciding between different data structures you're often faced with a situation like: X is faster at A, but slower at B; Y is the opposite. Which do you choose? I can't tell you, you have to know your intended use-cases. Will A dominate in your application, or B? This is the kind of thinking that scales, too. A database that's better at random access reads and writes is better for some applications than one that's optimized for sequential reads but lousy at random writes, so choose the suitable option based on a reasonable expectation of your system's needs. Study parallel programming (that is, actually having two or more threads/processes/whatevers running simultaneously). This will help you understand the limits of parallelism in speeding up programs, but also the potential for some programs. You'll learn about how to divide work across N processors, the limits of performance gains, and where you will get performance losses (critical ones: synchronization and communication). Study concurrency so you can see how to make use of asynchronous patterns. I don't mean strictly async/await style, just how to use whatever language/OS facilities to avoid unnecessary blocking. Like, if you're making a network query it can take seconds or longer to resolve (depending on location, size of the data, remote server load, etc.). Your program shouldn't wait if it still has other things it can do (if it doesn't, then let it wait). Use a profiler, study your real systems and see where they spend their time. What functions get called the most, what functions or syscalls consume the most time. Is there a file/resource that's constantly being used? I once sped up a program by around 60x just by realizing that the previous coder was repeatedly reading from the same file. It wasn't needed, he could have read from it once (optimal), or at least cached it in memory (though memory wasn't so plentiful then, it would've been a tight fit on a mid-00s desktop, but fine today). Apply the ideas above to analyzing it and improving it.

salawat 4y ago
On parallelism: Sufficient attentiin applied to parallelism/concurrency, is indistinguishable from management, without any of the benefits or recognition, or most other practitioners bothering to even try to to grok it. Does that mean you shouldn't do it? No. However, there comes a point where if you breach that barrier, you will discover that the fundamental problems of today arise out of willingness to inflict pain stemming from the component actors in the Human Virtual Machine.

blt 4y ago

> if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning? Learn how to use a sampling profiler.

RedShift1 4y ago

Use an old computer. Slow operations are enlarged which makes it more obvious where the problem lies. I have an old HP EliteBook 6930p with a Core 2 Duo P8700 CPU from 2008. I did put an SSD in it because Windows 10 is unusable without it.

Narishma 4y ago
Using an SSD hides a lot of IO-related performance issues though. I suspect it's why Windows 10 is super slow on an HDD.