7 ms·

Yes I absolutely agree. I posted a blog post near the start of the year looking at how much performance automerge (written in javascript using immutablejs) leav

Yes I absolutely agree. I posted a blog post near the start of the year looking at how much performance automerge (written in javascript using immutablejs) leaves on the table. At the time, automerge took 5 minutes to run a benchmark, and my own code could do the same thing in 0.06 seconds. Since then I’ve sped up my code by another order of magnitude - it can now run the same benchmark in 0.005s. (4.9ms). I don’t want to badmouth automerge - they’re working on it and have made big improvements this year. But that ratio - 5min:5ms - gives you a sense of the mountain of inefficiency we live with on modern computers with a lot of modern software. Modern CPUs and ram are obscenely, incomprehensibly fast. With the exception of a few very specific tasks, you should never be waiting for a modern computer to do anything. We sent men to the moon in fewer cycles than it takes for modern windows to boot, by several orders of magnitude. Complaining about the global chip shortage is a bit like a hoarder complaining about a room shortage in their overflowing house. (With carve outs in my scorn for video editing, AAA game engines, AI, etc)

bob1029 5y ago
> Modern CPUs and ram are obscenely, incomprehensibly fast. With the exception of a few very specific tasks, you should never be waiting for a modern computer to do anything. I really wish we could drive this point home for everyone. The amount of work you can get out of 1 x86 core is insane if you know how to feed it properly. The upper bound is somewhere around 500 million transactions per second per core. I have personally built systems that can achieve somewhere around 14 million per second. If you can reliably handle the entire serialized workload on a single thread you start to retool for the idea that you can fit the entire business in 1 box. Being able to run everything out of a single computer is an incredible virtue, not some curse that must be cast to the clouds.
- DaiPlusPlus 5y ago
  > 500 million transactions per second per core. An instruction is not a transaction. Commodity x86/x64 systems do not implement transactional memory or any model of computation that could be reasonably described as transactional as far as I'm aware.
  - bob1029 5y ago
    I'm not referring to individual instructions. I'm talking about actual useful changes to business state.
  - ben-schaaf 5y ago
    500 million instructions per second can be achieved on a pentium pro from 1995, modern CPUs run many orders of magnitude faster than that.
- sdevonoes 5y ago
  > Being able to run everything out of a single computer is an incredible virtue, not some curse that must be cast to the clouds. Isn't the problem then: what happens when take computer goes down? I'm also on the side of "know your hardware", but the topic of resilience is a different one. Also, perhaps you didn't mean "single computer" in the literal way: it's not incompatible with avoid the cloud, though.
  - withinboredom 5y ago
    I built a production grade cluster for fun. It’s only three nodes… without anything installed on my 40 core cluster, except k8s and longhorn, it’s idling at 30% cpu usage. When I started up a managed cluster on Azure, I was throwing hundreds of euros at their logging storage just idling, so I noped to bare metal.
    - KronisLV 5y ago
      This mirrors my experience with K8s, though one can suggest that the software running on the cluster also played a part in it. Have you looked into K3s or other lightweight distros of Kubernetes, though? https://k3s.io/ https://k3s.io/ That's what i turned to when i needed to learn the Kubernetes API and explore running clusters, but didn't like my small VPSes (~4 GB of RAM) failing to launch anything because of the frankly insane resource usage of full K8s clusters.
  - bob1029 5y ago
    I did mean single computer in terms of 1 active primary machine at any given time. Negotiate the advantages of this architecture with the business. For the systems I've built, synchronous log replication is a natural extension of the performance abstraction, and provides fairly robust protection when some manual intervention is allowed.
  - gjulianm 5y ago
    A good thing with "single computer" is that you have less complexity going into it so failures will be less frequent. For a lot of cases, that will be enough. It's also easier to build a resilient system starting from a single machine than from a cluster: less components to replicate to ensure they're redundant.
- moeris 5y ago
  Most applications are IO bound, not CPU bound. So wasted cycles aren't always that big of a deal.
  - BeefWellington 5y ago
    While this is most certainly true, I'm willing to bet the ratio between them has dropped substantially in recent years.
  - NohatCoder 5y ago
    Most applications do far more IO than what is necessary for the task. Anyway with a modern SSD I don't think you are right. My disk can do something like 3 GB/s and 100k random accesses per second. At that rate most desktop programs should have the binary and all dependencies loaded to memory in approximately 0.1s. That seemingly doesn't stop a lot of them from taking several seconds to boot. Likewise games, despite usually consuming less than 10 GB of video and main memory combined, can take much longer than 3.3s to load.
    - kaba0 5y ago
      Though that includes possibly decompression, parsing, and more importantly loading data to the GPU and even compiling shaders.
      - NohatCoder 5y ago
        Decompression is fast if you do it right (using anything from the Oodle pack is a good first step towards doing it right). Hopefully a game doesn't need to parse a whole lot of things, however stuff needs to appear in memory, it can be stored that way on disk, possibly compressed with aforementioned fast compression libraries. There is nothing special about loading data onto the graphics card, if you know what you are doing it is just copying data, and no point on the path is slower than disk. I know there are games that somehow rely on thousands of shaders. Like most games you can just not do that, or you can store the compiled shaders so that the compilation only needs to be done on first run.
  - 5y ago
- sokoloff 5y ago
  > The upper bound is somewhere around 500 million transactions per second per core. What’s the definition of transaction here? That’s a transaction for every ~6 (peak ~10) clock cycles. https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf
  - bob1029 5y ago
    Transaction in this context being "Place order", "Cancel order", "Update user settings", etc. We need to remember there isn't a 1:1 relationship between instruction/clock rate and the number of actual instructions that are retired per unit time. Pipelining and ILP allow for speedups that range anywhere from 1-100x depending on the specific flavor of application.
deleted 5y ago
[deleted]
BeefWellington 5y ago
There's an underlying element that isn't explored in this enough: constraint is many times useful (rephrasing of Necessity is the mother of all invention). The prevailing sentiment these days seems to be that developers don't need to reason about performance "because we can just scale it." It might be useful, or at least informative, to start with "make your app run well on a Raspberry PI" or equivalent low-power system, and then consider your performance from there.
hinkley 5y ago
I can’t tell you how many times I’ve worked with people who look at a flat perf chart and decide that since they can’t see a tall tent pole there’s nothing more to be done. If the business insists that customers are complaining, they just show the charts as if that explains anything. Charts don’t absolve you. They just mean you’ve given up without learning much at all. Flame charts might have helped a little, but it’s still largely the same problem. On my Google interview, after I knew they weren’t interested, they asked me what the hardest thing is in software, I told them one of my more controversial beliefs because 1) fuck it, and 2) sometimes these seeds grow in the most unlikely of places. And that is that managing the C constant is the hardest thing we do. Paraphrasing an old, old manager: anyone can do the first order of magnitude, the second is hard, and the third is beyond many people. It’s not an intellectual process. It’s discipline. Pure, concentrated discipline to keep doing the hard work to figure out all of the tiny XY problems in your code that have a performance penalty even if it’s just a factor of two or three. And once you’ve found them, care and organization to not create huge externalities for others. Refactoring doesn’t keep you from making regressions, it just saves you 9 times out of ten, 99 out of 100 at the most. You’re gonna break an egg or two per year. I have a sense that it is very rare for people who start out writing very inefficient code to have a much of plan to make it faster. Some don’t even have a plan to make it right, let alone make it fast. If things are as you say, then more than likely automerge will need new leadership from new contributors, and/or it will always lag behind by an order of magnitude.
- KronisLV 5y ago
  > If the business insists that customers are complaining, they just show the charts as if that explains anything. Charts don’t absolve you. They just mean you’ve given up without learning much at all. This mirrors my own experience in the industry. When clients complain that loading some data in a CRUD app takes them upwards of 6 seconds in a bespoke ERP system, they're met by the blank stares of some devs. A cursory review of the code reveals that what could have been a single SQL query is instead a bunch of service calls which ends up being a textbook example of the N+1 problem. And when asked about this, the devs respond with: "But it's easier to use service calls to get the data because of code reuse, working with complex SQL queries is harder." Well, you picking the easier way out caused the performance to be bad in this case, especially because of your needless queries lead to degraded performance for all of the users in the app. But how could you even get to that point? Because the other devs who did the code review were struggling to even review the overcomplicated business logic in the new code. Because they didn't care enough to ask about the architecture and the N+1 problem, or because they simply didn't see it in the overcomplicated codebase. Because the merge/pull request description is literally blank because no one cares about keeping track of ADRs (architecture design records) or even the historical context for certain changes. Because there has been very little care put into load testing in any capacity so far, because the system doesn't lend itself nicely to testing or even setting up new environments, any attempt to do which is met with resistance from ops. New features get mostly prioritized over maintenance, addressing technical debt or solving these longstanding issues, both from the client company and management side. And when things inevitably do break, the devs are blamed for seemingly not doing this all on their own time. That disconnect between what should be done if anyone actually cared about software engineering and what is actually done in the industry irks me greatly. I hate the short term thinking that makes systems perform badly in the long term and hard to maintain. That is not sustainable in any way whatsoever. Luckily, i've basically just said "no" to the above and for the past N months have been working on modernizing the app, adding proper APM instrumentation and monitoring, improving environment setup whenever possible (enterprise DB is still an issue, everything else currently being managed with Ansible) and introducing containers for easier runtime management and health checks with restarts/load balancing, while also writing unit tests (some of my new code has 100% coverage, actually) and addressing technical debt. At this point, i don't care if it earns me scorn, pressure or if i get fired because of it down the road - i'm an engineer (with a degree that says exactly that) and that's what i'll do, asking neither permission, nor forgiveness (while clearly communicating what i'm doing as a matter of fact). Sure, there are times when you have to cut corners and ship things or help others with sub-optimal solutions, but i wish that more people in the industry didn't waiver under pressure from management and didn't choose to lie about estimates just to please them.
rmbyrro 5y ago
Would you suggest any resources to learn the way you point out (more efficient use of hardware)? Books, courses, a roadmap, topics I should learn...
- bob1029 5y ago
  If you are looking for a deep take on what the hardware actually wants us to do with it, you could start here: https://lmax-exchange.github.io/disruptor/disruptor.html https://lmax-exchange.github.io/disruptor/disruptor.html Understanding the performance journey that these financial exchanges went through has saved me a lot of time and frustration when attempting to build similar high-performance software.

bob1029 5y ago

> Modern CPUs and ram are obscenely, incomprehensibly fast. With the exception of a few very specific tasks, you should never be waiting for a modern computer to do anything. I really wish we could drive this point home for everyone. The amount of work you can get out of 1 x86 core is insane if you know how to feed it properly. The upper bound is somewhere around 500 million transactions per second per core. I have personally built systems that can achieve somewhere around 14 million per second. If you can reliably handle the entire serialized workload on a single thread you start to retool for the idea that you can fit the entire business in 1 box. Being able to run everything out of a single computer is an incredible virtue, not some curse that must be cast to the clouds.

DaiPlusPlus 5y ago
> 500 million transactions per second per core. An instruction is not a transaction. Commodity x86/x64 systems do not implement transactional memory or any model of computation that could be reasonably described as transactional as far as I'm aware.
- bob1029 5y ago
  I'm not referring to individual instructions. I'm talking about actual useful changes to business state.
- ben-schaaf 5y ago
  500 million instructions per second can be achieved on a pentium pro from 1995, modern CPUs run many orders of magnitude faster than that.
sdevonoes 5y ago
> Being able to run everything out of a single computer is an incredible virtue, not some curse that must be cast to the clouds. Isn't the problem then: what happens when take computer goes down? I'm also on the side of "know your hardware", but the topic of resilience is a different one. Also, perhaps you didn't mean "single computer" in the literal way: it's not incompatible with avoid the cloud, though.
- withinboredom 5y ago
  I built a production grade cluster for fun. It’s only three nodes… without anything installed on my 40 core cluster, except k8s and longhorn, it’s idling at 30% cpu usage. When I started up a managed cluster on Azure, I was throwing hundreds of euros at their logging storage just idling, so I noped to bare metal.
  - KronisLV 5y ago
    This mirrors my experience with K8s, though one can suggest that the software running on the cluster also played a part in it. Have you looked into K3s or other lightweight distros of Kubernetes, though? https://k3s.io/ https://k3s.io/ That's what i turned to when i needed to learn the Kubernetes API and explore running clusters, but didn't like my small VPSes (~4 GB of RAM) failing to launch anything because of the frankly insane resource usage of full K8s clusters.
- bob1029 5y ago
  I did mean single computer in terms of 1 active primary machine at any given time. Negotiate the advantages of this architecture with the business. For the systems I've built, synchronous log replication is a natural extension of the performance abstraction, and provides fairly robust protection when some manual intervention is allowed.
- gjulianm 5y ago
  A good thing with "single computer" is that you have less complexity going into it so failures will be less frequent. For a lot of cases, that will be enough. It's also easier to build a resilient system starting from a single machine than from a cluster: less components to replicate to ensure they're redundant.
moeris 5y ago
Most applications are IO bound, not CPU bound. So wasted cycles aren't always that big of a deal.
- BeefWellington 5y ago
  While this is most certainly true, I'm willing to bet the ratio between them has dropped substantially in recent years.
- NohatCoder 5y ago
  Most applications do far more IO than what is necessary for the task. Anyway with a modern SSD I don't think you are right. My disk can do something like 3 GB/s and 100k random accesses per second. At that rate most desktop programs should have the binary and all dependencies loaded to memory in approximately 0.1s. That seemingly doesn't stop a lot of them from taking several seconds to boot. Likewise games, despite usually consuming less than 10 GB of video and main memory combined, can take much longer than 3.3s to load.
  - kaba0 5y ago
    Though that includes possibly decompression, parsing, and more importantly loading data to the GPU and even compiling shaders.
    - NohatCoder 5y ago
      Decompression is fast if you do it right (using anything from the Oodle pack is a good first step towards doing it right). Hopefully a game doesn't need to parse a whole lot of things, however stuff needs to appear in memory, it can be stored that way on disk, possibly compressed with aforementioned fast compression libraries. There is nothing special about loading data onto the graphics card, if you know what you are doing it is just copying data, and no point on the path is slower than disk. I know there are games that somehow rely on thousands of shaders. Like most games you can just not do that, or you can store the compiled shaders so that the compilation only needs to be done on first run.
- 5y ago
sokoloff 5y ago
> The upper bound is somewhere around 500 million transactions per second per core. What’s the definition of transaction here? That’s a transaction for every ~6 (peak ~10) clock cycles. https://www.agner.org/optimize/instruction_tables.pdf https://www.agner.org/optimize/instruction_tables.pdf
- bob1029 5y ago
  Transaction in this context being "Place order", "Cancel order", "Update user settings", etc. We need to remember there isn't a 1:1 relationship between instruction/clock rate and the number of actual instructions that are retired per unit time. Pipelining and ILP allow for speedups that range anywhere from 1-100x depending on the specific flavor of application.

deleted 5y ago

[deleted]

BeefWellington 5y ago

There's an underlying element that isn't explored in this enough: constraint is many times useful (rephrasing of Necessity is the mother of all invention). The prevailing sentiment these days seems to be that developers don't need to reason about performance "because we can just scale it." It might be useful, or at least informative, to start with "make your app run well on a Raspberry PI" or equivalent low-power system, and then consider your performance from there.

hinkley 5y ago

I can’t tell you how many times I’ve worked with people who look at a flat perf chart and decide that since they can’t see a tall tent pole there’s nothing more to be done. If the business insists that customers are complaining, they just show the charts as if that explains anything. Charts don’t absolve you. They just mean you’ve given up without learning much at all. Flame charts might have helped a little, but it’s still largely the same problem. On my Google interview, after I knew they weren’t interested, they asked me what the hardest thing is in software, I told them one of my more controversial beliefs because 1) fuck it, and 2) sometimes these seeds grow in the most unlikely of places. And that is that managing the C constant is the hardest thing we do. Paraphrasing an old, old manager: anyone can do the first order of magnitude, the second is hard, and the third is beyond many people. It’s not an intellectual process. It’s discipline. Pure, concentrated discipline to keep doing the hard work to figure out all of the tiny XY problems in your code that have a performance penalty even if it’s just a factor of two or three. And once you’ve found them, care and organization to not create huge externalities for others. Refactoring doesn’t keep you from making regressions, it just saves you 9 times out of ten, 99 out of 100 at the most. You’re gonna break an egg or two per year. I have a sense that it is very rare for people who start out writing very inefficient code to have a much of plan to make it faster. Some don’t even have a plan to make it right, let alone make it fast. If things are as you say, then more than likely automerge will need new leadership from new contributors, and/or it will always lag behind by an order of magnitude.

KronisLV 5y ago
> If the business insists that customers are complaining, they just show the charts as if that explains anything. Charts don’t absolve you. They just mean you’ve given up without learning much at all. This mirrors my own experience in the industry. When clients complain that loading some data in a CRUD app takes them upwards of 6 seconds in a bespoke ERP system, they're met by the blank stares of some devs. A cursory review of the code reveals that what could have been a single SQL query is instead a bunch of service calls which ends up being a textbook example of the N+1 problem. And when asked about this, the devs respond with: "But it's easier to use service calls to get the data because of code reuse, working with complex SQL queries is harder." Well, you picking the easier way out caused the performance to be bad in this case, especially because of your needless queries lead to degraded performance for all of the users in the app. But how could you even get to that point? Because the other devs who did the code review were struggling to even review the overcomplicated business logic in the new code. Because they didn't care enough to ask about the architecture and the N+1 problem, or because they simply didn't see it in the overcomplicated codebase. Because the merge/pull request description is literally blank because no one cares about keeping track of ADRs (architecture design records) or even the historical context for certain changes. Because there has been very little care put into load testing in any capacity so far, because the system doesn't lend itself nicely to testing or even setting up new environments, any attempt to do which is met with resistance from ops. New features get mostly prioritized over maintenance, addressing technical debt or solving these longstanding issues, both from the client company and management side. And when things inevitably do break, the devs are blamed for seemingly not doing this all on their own time. That disconnect between what should be done if anyone actually cared about software engineering and what is actually done in the industry irks me greatly. I hate the short term thinking that makes systems perform badly in the long term and hard to maintain. That is not sustainable in any way whatsoever. Luckily, i've basically just said "no" to the above and for the past N months have been working on modernizing the app, adding proper APM instrumentation and monitoring, improving environment setup whenever possible (enterprise DB is still an issue, everything else currently being managed with Ansible) and introducing containers for easier runtime management and health checks with restarts/load balancing, while also writing unit tests (some of my new code has 100% coverage, actually) and addressing technical debt. At this point, i don't care if it earns me scorn, pressure or if i get fired because of it down the road - i'm an engineer (with a degree that says exactly that) and that's what i'll do, asking neither permission, nor forgiveness (while clearly communicating what i'm doing as a matter of fact). Sure, there are times when you have to cut corners and ship things or help others with sub-optimal solutions, but i wish that more people in the industry didn't waiver under pressure from management and didn't choose to lie about estimates just to please them.

rmbyrro 5y ago

Would you suggest any resources to learn the way you point out (more efficient use of hardware)? Books, courses, a roadmap, topics I should learn...

bob1029 5y ago
If you are looking for a deep take on what the hardware actually wants us to do with it, you could start here: https://lmax-exchange.github.io/disruptor/disruptor.html https://lmax-exchange.github.io/disruptor/disruptor.html Understanding the performance journey that these financial exchanges went through has saved me a lot of time and frustration when attempting to build similar high-performance software.