7 ms·
I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all.
by JackSlateur 1y ago
I think the author is mislead
Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all.
You do your test, all good, you can roll in production !
But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything about that, and "your tests were green".
Algorithms are the backbone tools of programming. Knowing them help us, ignoring them burdens us.
- wubrr 1y agoI mean, in your example you just have an incomplete test suite. (Though writing a complete one is often unrealistic) While understanding algorithms and data structures is important, the only way you really know how well it works, and how well it's implemented is by thoroughly testing it. There are an infinite amount of clever algorithms out there with terrible implementations. You need both.
- JackSlateur 1y agoTesting concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;
- wubrr 1y ago> Testing concurrency is extremely hard Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder. > I truly wonder if there is real world tests around this Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and the systems involved is required to write a proper test suite. > For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Take those queries and run them 1000x+ times concurrently in a loop. That will catch most common issues. If you want to go a step further you can build a setup to execute your queries in any desired order.
- sarchertech 1y agoI’ve never worked somewhere (in 20 years from big tech companies to small startups) that was generally and reliably testing for concurrency bugs. And I’ve seen dozens of bugs caused by people assuming that transactions (with the default isolation level) protect against race conditions.
- wubrr 1y agoEvery place I worked at, that had any kind of reliable, high-throughput concurrent system had an extensive suite of concurrent tests. https://github.com/postgres/postgres/tree/master/src/test/isolation https://github.com/postgres/postgres/tree/master/src/test/is... https://muratbuffalo.blogspot.com/2023/08/distributed-transactions-at-scale-in.html https://muratbuffalo.blogspot.com/2023/08/distributed-transa... https://learn.microsoft.com/en-us/archive/msdn-magazine/2008/june/tools-and-techniques-to-identify-concurrency-issues https://learn.microsoft.com/en-us/archive/msdn-magazine/2008... https://go.dev/blog/synctest https://go.dev/blog/synctest https://learntla.com/core/concurrency.html https://learntla.com/core/concurrency.html
- sarchertech 1y ago> Every place I worked at, that had any kind of reliable, high-throughput concurrent system Pretty much anyone with high throughput is running a high throughput concurrent system, and very few companies have an extensive suite of concurrency tests unless you just mean load tests (that aren’t setup to catch race conditions). The “reliable” part of that statement might be doing a lot of heavy lifting depending on what exactly you mean by that.
- wubrr 1y agoI gave you several concrete examples. Your claims of 'very few companies have...' aren't very convincing, and the apparent popularity of concurrency testing isn't really a strong argument for or against it's effectiveness or do-ability.
- hxtk 1y agoI’ve long wished for an SQL error model: given a schema, query, and transaction isolation mode, what errors are theoretically possible? I have a hard time answering this for Postgres, which disappoints me because I don’t see any reason it sounds very easy to answer, like there could be an extension to EXPLAIN that would dry run the query and list all the error states reachable.
- jiggawatts 1y agoComputer Science is incredibly immature. Many of its "founding fathers" are still alive! Something akin to this that blew my mind recently was an IDE for a functional language that used typed holes, the programming equivalent of a semiconductor electron "vacancy", a quasi-particle with real properties that is actually just the lack of a particle. The concept was that if you delete (or haven't yet typed) some small segment out of an otherwise well-typed program, the compiler can figure out the type that the missing part must have. This can be used for rapid development because many such "holes" can only have one possible type. This kind of mechanistic development with tool assistance is woefully under-developed.
- lucianbr 1y agohttps://jepsen.io/ https://jepsen.io/
- toolslive 1y agoThere are "lightweight formal methods". Most problems can be produced via small models. Tools like alloy are built around this idea. (IIRC alloy was used to show that a famous DHT had issues with the churn protocol) https://en.wikipedia.org/wiki/Alloy_(specification_language) https://en.wikipedia.org/wiki/Alloy_(specification_language)
- terpimost 1y agohttps://antithesis.com/ https://antithesis.com/ was made to deal with this. You can think of its as a fuzzing but it has overall determinism for the whole system, so there is a time travel and interactive debugging.
- eevmanu 1y agodeterministic simulation testing[1] (DST) [1] https://notes.eatonphil.com/2024-08-20-deterministic-simulation-testing.html https://notes.eatonphil.com/2024-08-20-deterministic-simulat...
- hinkley 1y agoMediocre testing can also lead to a situation where there is friction for improvement because the tests are brittle and coupled (with each other and the misfeature you’re interested in fixing). I like a more uniform distribution in my testing efforts. Start earlier, end later than most, and it’s experiences like this that inform that preference. And also production bugs in code with supposed 100% test coverage.
- sarchertech 1y ago> Mediocre testing can also lead to a situation where there is friction for improvement because the tests are brittle and coupled This is very very common among inexperienced devs and in immature organizations that think that more tests necessarily means better.
- hinkley 1y agoAnd devs who think they are experienced by disappear when the testing gets tough.
- themafia 1y agoMacro Pierre White says "perfection is lots of little things done well." Which is something I've always agreed with, so, I never understand articles that seek to eschew an important part of releasing software because they believe their approach elsewhere is enough to overcome these intentionally suboptimal choices.
- yakshaving_jgt 1y agoAlmost all of professional software should be intentionally suboptimal. This is what we mean when we say that premature optimisation is the root of all evil.
- FridgeSeal 1y agoWe ought to ban people saying that quote, due to the way it has been abused to avoid considering performance _at all_. “Intentionally suboptimal” is also a strange way of phrasing it, as it makes it sound a bit like you’re intentionally building something bad, as opposed to “only as good as it needs to be”.
- yakshaving_jgt 1y agoIn general I avoid considering performance at all. Instead, I focus on adding testing and instrumentation. When the telemetry tells me I have a performance problem, then I can solve that part while being confident that the performance improvement doesn't change the functionality because I first invested in tests.