3 ms·
I would add that software infrastructure can run incredibly fast and scale incredibly well on modern hardware if you're a bit careful about resource usage. Tra
by BenoitEssiambre 5y ago
I would add that software infrastructure can run incredibly fast and scale incredibly well on modern hardware if you're a bit careful about resource usage.
Traditional relational DBs like Postgresql are very scalable on modern hardware. If you take the time to craft a normalized schema with low redundancy being careful about keeping data small, you can achieve performance and resource efficiency and cache efficiency hundreds of time better than bloated distributed document nosql based systems. You can also get better transactional integrity, reduce reliance on queues (use load balancers instead) and get more instant and more atomic error propagation and edge case resolution. You can really build something lean and mean to a point that would be physically impossible in distributed systems or systems that involve multiple hops over potentially congested networks (as Admiral Grace Hopper likes to remind us, distance matters in computing https://www.youtube.com/watch?v=9eyFDBPk4Yw https://www.youtube.com/watch?v=9eyFDBPk4Yw). As far as I know, normalized relational databases are still the best at efficiently using multiple levels of physically near caches to their full potential. Most applications don't need to scale horizontally and can easily fit on single servers (that can scale to dozens of cores and terabytes of ram).
I hear arguments that it's the engineers that are expensive so it's ok to be wasteful with hardware if it saves engineering time but in my experience the type of engineers that are good at optimizations are often good at engineering in general and the ones that forget to think about efficiency are also sloppy with other things. It doesn't mean you have to write your code in C. Very fast code can be written in high level languages with the proper skills (See projects like Fastify).
- nikanj 5y agoUnfortunately in a lot of cases, it’s _our_ engineers are expensive so we are wasting _your_ CPU/RAM/etc
- bob1029 5y ago> Very fast code can be written in high level languages with the proper skills I sometimes wonder why we don't have mandatory coursework that demonstrates the upper bound of what a modern x86 system is capable of in practical terms. If developers understood just how much perf they were leaving on the plate, they would likely self-correct out of shame. Latency is the ultimate devil and we need to start burning that into the brains of developers. The moment you shard a business system to more than 1 computer, you enter into hell. We should be trying to avoid this fate, not embrace it. We basically solved the question "what's the fastest way to synchronize work between threads" in ~2010 with the advent of the LMAX Disruptor. But, for whatever reason this work has been relegated to the dark towers of fintech wizardry rather than being perma-stickied on HN. Systems written in C#10 on .NET6 which leverage a port of this library can produce performance figures that are virtually impossible to meet in any other setting (at least in a safe/stable way). This stuff is not inaccessible at all. It is just unpopular. Which is a huge shame. There is so much damage (in a good way) developers could do with these tools and ideologies if they could find some faith in them.