4 ms·
I'm not disagreeing with anything you said here - reread my comment. Sometimes you want to use the sophisticated shiny new tech because you actually need it. H
by sophacles 2y ago
I'm not disagreeing with anything you said here - reread my comment.
Sometimes you want to use the sophisticated shiny new tech because you actually need it. Here's a recent example from a real situation:
The linux kernel (a boring tech these days) has a great networking stack. It's choking on packets that need to be forwarded, and you've already tuned all the queues and the cpu affinities and timers and polling. Do you -
a) buy more servers and network gear to spread your packets across more machines? (boring and expensive and introduces new ongoing costs of maintenance, datacenter costs, etc).
b) Write a kernel module to process your packets more efficiently? (a boring, well known solution, introduces engineer costs to make and maintain as well as downtime because the new shiny module is buggy?)
c) Port your whole stack to a different OS (risky, but choosing a different boring stack should suffice... if youre certain that it can handle the load without kernel code changes/modules).
d) Write a whole userspace networking system (trendy and popular - your engineers are excited about this, expensive in eng time, risks lots of bugs that are already solved by the kernel just fine, have to re-invent a lot of stuff that exists elsewhere)
e) Use ebpf to fast path your packets around the kernel processing that you don't need? (trendy and popular - your engineers are excited about this, inexpensive relative to the other choices, introduces some new bugs and stability issues til the kinks are worked out)
We sinned and went with (e). That new-fangled tech met our needs quite well - we still had to buy more gear but far less than projected before we went with (e). We're actually starting to reach limits of ebpf for some of our packet operations too so we've started looking at (d) which has come down in costs and risk as we understand our product and needs better.
I'm glad we didn't go the boring path - our budget wasn't eaten up with trying to make all that work and we could afford to build features our customers buy instead.
We also use postgres to store a bunch of user data. I'm glad we went the boring path there - it just works and we don't have to think about it, and that lack of attention has afforded us the chance to work on features customers buy instead.
The point isn't "don't choose boring". It's: blindly choosing boring instead of evaluating your actual needs and options from a knowledgeable place is unwise.
- zozbot234 2y agoNone of these seem all that 'trendy' to me. The real trendy approach would be something like leaping directly to a hybrid userspace-kernelspace solution using something like https://github.com/CloudNativeDataPlane/cndp https://github.com/CloudNativeDataPlane/cndp and/or the https://www.kernel.org/doc/html/latest/networking/af_xdp.html https://www.kernel.org/doc/html/latest/networking/af_xdp.htm... addressing that the former is built on. Very interesting stuff, don't get me wrong there - but hardly something that can be said to have 'stood the test of time' like most boring tech has. (And I would include things like eBPF in that by now.)
- sophacles 2y agoI have similar examples from other projects of using io_uring and af_xdp with similar outcomes. In 2020 when the ebpf decision was made it was pretty new an trendy still too... in a few cases each of these choices required us to wait for deployment until some feature we chose to depend on landed in a mainline kernel. Things move a bit slower that far down the stack so new doesn't mean "the js framework of the week", but it's still the trendy unproven thing vs the well-known path. The point is still: evaluate the options for real - using the new thing because it's new and exicting is equally as foolish as use the boring thing because it's well-proven... if those are your main criteria.