3 ms·
Speed is a tricky thing in a complex application. You are ultimately going to be forced to trade latency for horizontal scalability in non-trivial applications
by bob1029 6y ago
Speed is a tricky thing in a complex application. You are ultimately going to be forced to trade latency for horizontal scalability in non-trivial applications with lots of shared state which must also be consumed globally.
You can cheat in some weird and fun ways though. For instance, if you say "no user of this system will ever be more than 50ms away", you get to play some really interesting games with vertical scaling and consolidation of compute capability in an all-in way. I.e. server-side technologies ran out of a single datacenter near the userbase.
If your latency domain fits it, something like Blazor server side can be an incredible experience for your users. First load is almost immediate because there's virtually no client code. Everything is incremental from there. If you are within 50ms of the server, UI feels instant in my experience. The nature of how applications are developed with this tech means that if your business services are completing requests within the performance budget, you can be almost certain the end user will see the same.
Going to the bottom of the rabbit hole, understanding how NUMA impacts performance can make 5+ orders of magnitude difference in latency and throughput. Letting a thread warm up on a hot path and keeping it fed with well-structured data is foundational to ultra-low-latency processing.
You can handle well over a million events per second on a single thread on any recent PC using techniques such as LMAX disruptor combined with a web hosting technology like Kestrel. The difference between a class and a struct can be 10x if you get to that level of optimization. I measure user interactions in microseconds in my stack these days.
A millisecond is a fucking eternity. You shouldn't be doing a bunch of back and forth bullshit in that kind of latency domain. Stream client events to server as fast as possible, microbatch and process, prepare final DOM changeset and send to client all at once. How could any other client-server architecture be faster than this, especially if we are forced to care about a bucket of shared state?