4 ms·

> And that explain why companies do full rewrites of their code for performance: the effort needed to squeeze more performance from the existing code becomes to

> And that explain why companies do full rewrites of their code for performance: the effort needed to squeeze more performance from the existing code becomes too much and a complete rewrite is cheaper. The article provides reasons why optimization gets harder, but no arguments for why a rewrite is better. It's unclear whether the author is arguing for rewrites or whether they're simply pointing out why companies take them on. Arguably, though, companies taking on a full rewrite surely must have considered the cost of optimization (versus naively saying "the system is slow, replace it!"—though maybe some did). Rewrites are big, expensive, and time-consuming. It means new bugs and unknown unknowns, and no time to add features or fix bugs because you're busy rewriting functional code. It's a scapegoat for lack of improvement or progress. You shouldn't take one on lightly. At the same time, this post also neglects that some efficiency wins have little to do with the efficiency of the code, but rather the efficiency of the logic. An N+1 query in your application looks like your database is slow: you're wasting a ton of time sitting and waiting for your DB to return information! But the real problem is that you're repeatedly going back-and-forth to the database to query lots of little pieces of information that could have far more efficiently been queried all at once. > It is relatively easy to double the performance of an unoptimized piece of code, but much harder to multiply it by 10. You quickly hit walls that can be unsurmountable: the effort needed to double the performance again would just be too much. That's not really true, though. One bad SQL query can go from many seconds or minutes to milliseconds. One accidentally-quadratic algorithm can take orders of magnitude more time than a linear-time algorithm. One bad regexp can account for the majority of a request. Of course, as you fix the biggest performance problems, the only problems left are ones that are smaller than your biggest ones, so you'll have diminishing returns. But it also begs the question, what choices has your existing code made that makes it _ten times_ slower than you want it to be? In my experience, you're doing work synchronously that could have been put in a queue and worked on asynchronously. It's more often "you're doing more work than you should" or "you're being inefficient with the resources you have available" than "a specific piece of code is computationally inefficient".

nemothekid 4y ago
>The article provides reasons why optimization gets harder, but no arguments for why a rewrite is better. It's unclear whether the author is arguing for rewrites or whether they're simply pointing out why companies take them on. He didn't argue a rewrite is just "better"; his argument was that a rewrite was the only card of the table. The architecture was deficient and to get more performance you have to change the architecture, which means a rewrite. I tend to agree; I take the view that most engineers are smart, and compilers/interpreters/virtual machines are even smarter so most targeted optimizations aren't going to result in very much gain. A codebase full of N+1 queries or unindexed queries never cared about performance to begin with. For true gains, you will have to think about data which is the true bottleneck for most applications - getting data from memory, the disk or the network will be much longer that any instruction cycle. The way memory moves through your application is baked into your architecture and changing this will almost always involve a rewrite. To your final point, >In my experience, you're doing work synchronously that could have been put in a queue and worked on asynchronously. moving from a synchronous codebase to an async one almost always involves a rewrite.
- josephg 4y ago
  > I tend to agree; I take the view that most engineers are smart, and compilers/interpreters/virtual machines are even smarter so most targeted optimizations aren't going to result in very much gain. This hasn’t been my experience. As an example, I find there’s an awful lot of performance left on the table in most programs because of the sloppy way programmers use memory. Most programmers don’t think twice about allocating memory and pointer indirection, and most programs are full of it. But if you can refactor this stuff to use bigger objects and fewer allocations, and use object pools, arenas and inline allocation (smallvec, SSO and friends) where it’s appropriate you can usually improve your performance by several times in most “already optimized” programs. The performance comes from fewer malloc calls (malloc is expensive) and fewer cache misses (because locality improves). It’s like you say - Cache misses are super expensive on modern hardware. And you often don’t need a full rewrite to tweak this stuff - just some careful refactoring. I once saw a 20x performance uplift in a benchmark because we were using some tree structure where each leaf node only stored a single value. We replaced it with something that had arrays at the leaves and larger arrays in the internal nodes and performance skyrocketed. The compiler isn’t smart enough to suggest any of this stuff. If you use Box<X> in rust instead of X, it’ll happily give you a slow program. Slower than javascript in many cases. And the default Vec and String types in rust’s standard library allocate even if the contents would fit in a pointer. In javascript, Python and friends you can’t even implement a lot of low-allocation data structures because every list and object is inescapably a pointer to a heap object. This is why JS will never be as fast as C - you can’t write fast, nontrivial data structures. There’s a ceiling on the performance in languages like this - and if you need more performance than JS can give you, then a rewrite might be the right call. Another example: I had a bug yesterday where writing a 5mb JSON file took about 1 second. Turned out I wasn’t using a buffered writer. Wrapping my File in BufferedWriter::new() made the time taken drop from about 1 second to 0.01 seconds. The compiler isn’t very smart. By all means, rewrite your software to be fast. But there’s also usually big performance wins to be had in almost any program if you take the time to look.

nemothekid 4y ago

>The article provides reasons why optimization gets harder, but no arguments for why a rewrite is better. It's unclear whether the author is arguing for rewrites or whether they're simply pointing out why companies take them on. He didn't argue a rewrite is just "better"; his argument was that a rewrite was the only card of the table. The architecture was deficient and to get more performance you have to change the architecture, which means a rewrite. I tend to agree; I take the view that most engineers are smart, and compilers/interpreters/virtual machines are even smarter so most targeted optimizations aren't going to result in very much gain. A codebase full of N+1 queries or unindexed queries never cared about performance to begin with. For true gains, you will have to think about data which is the true bottleneck for most applications - getting data from memory, the disk or the network will be much longer that any instruction cycle. The way memory moves through your application is baked into your architecture and changing this will almost always involve a rewrite. To your final point, >In my experience, you're doing work synchronously that could have been put in a queue and worked on asynchronously. moving from a synchronous codebase to an async one almost always involves a rewrite.

josephg 4y ago
> I tend to agree; I take the view that most engineers are smart, and compilers/interpreters/virtual machines are even smarter so most targeted optimizations aren't going to result in very much gain. This hasn’t been my experience. As an example, I find there’s an awful lot of performance left on the table in most programs because of the sloppy way programmers use memory. Most programmers don’t think twice about allocating memory and pointer indirection, and most programs are full of it. But if you can refactor this stuff to use bigger objects and fewer allocations, and use object pools, arenas and inline allocation (smallvec, SSO and friends) where it’s appropriate you can usually improve your performance by several times in most “already optimized” programs. The performance comes from fewer malloc calls (malloc is expensive) and fewer cache misses (because locality improves). It’s like you say - Cache misses are super expensive on modern hardware. And you often don’t need a full rewrite to tweak this stuff - just some careful refactoring. I once saw a 20x performance uplift in a benchmark because we were using some tree structure where each leaf node only stored a single value. We replaced it with something that had arrays at the leaves and larger arrays in the internal nodes and performance skyrocketed. The compiler isn’t smart enough to suggest any of this stuff. If you use Box<X> in rust instead of X, it’ll happily give you a slow program. Slower than javascript in many cases. And the default Vec and String types in rust’s standard library allocate even if the contents would fit in a pointer. In javascript, Python and friends you can’t even implement a lot of low-allocation data structures because every list and object is inescapably a pointer to a heap object. This is why JS will never be as fast as C - you can’t write fast, nontrivial data structures. There’s a ceiling on the performance in languages like this - and if you need more performance than JS can give you, then a rewrite might be the right call. Another example: I had a bug yesterday where writing a 5mb JSON file took about 1 second. Turned out I wasn’t using a buffered writer. Wrapping my File in BufferedWriter::new() made the time taken drop from about 1 second to 0.01 seconds. The compiler isn’t very smart. By all means, rewrite your software to be fast. But there’s also usually big performance wins to be had in almost any program if you take the time to look.