8 ms·

Testing is better than data structures and algorithms

188 points by rsyring 1y ago

wjrb 1y ago
Are there any resources out there that anyone can recommend for learning testing in the way the author describes? In-the-trenches experience (especially "good" or "doing it right" experience) can be hard to come by; and why not stand on the shoulders of giants when learning it the first time?
- cogman10 1y ago
  Resources, none that I'm aware of. I generally think this is an OK way to look at testing [1], though I think it goes too far if you completely adopt their framework. The boil down the tests I like to see. Structure them with "Given/when/then" statements. You don't need a framework for this, just make method calls with whatever unit test framework you are using. Keep the methods small, don't do a whole lot of "then"s, split that into multiple tests. Structure your code so that you aren't testing too deep. Ideally, you don't need to stand up your entire environment to run a test. But do write some of those tests, they are important for catching issues that can hide between unit tests. [1] https://cucumber.io/docs/bdd/ https://cucumber.io/docs/bdd/
- Jtsummers 1y ago
  Working Effectively with Legacy Code by Michael Feathers. It spends a lot of time on how to introduce testability into existing software systems that were not designed for testing. Property-Based Testing with PropEr, Erlang, and Elixir by Fred Hebert. While a book about a particular tool (PropEr) and pair of languages (Erlang and Elixir), it's a solid introduction to property-based testing. The techniques described transfer well to other PBT systems and other languages. Test-Driven Development by Kent Beck. https://www.fuzzingbook.org/ https://www.fuzzingbook.org/ by Zeller et al. and https://www.debuggingbook.org/ https://www.debuggingbook.org/ by Andreas Zeller. The latter is technically about debugging, but it has some specific techniques that you can incorporate into how you test software. Like Delta Debugging, also described in a paper by Zeller et al. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=988498 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=988.... I'm not sure of other books I can recommend, the rest I know is from learning on the job or studying specific tooling and techniques.
  - GrumpyYoungMan 1y ago
    TDD is a development methodology, not a testing methodology. The main thing it does is check whether the developer implemented what they thought they should be implementing, which is not necessarily what the spec actually says to implement or what the end user expects.
    - Jtsummers 1y ago
      It's still a useful technique and way to apply testing to development. But yes, it's not the best resource in telling you what tests to write, more about how they can be applied effectively. Which is a skill that seems absent in many professionals.
- fuzztester 1y ago
  The Art of Software Testing. New York: Wiley, 1979 The Art of Software Testing, Second Edition. with Tom Badgett and Todd M. Thomas, New York: Wiley, 2004. It is by Glenford Myers (and others). https://en.m.wikipedia.org/wiki/Glenford_Myers https://en.m.wikipedia.org/wiki/Glenford_Myers From the top of that page: [ Glenford Myers (born December 12, 1946) is an American computer scientist, entrepreneur, and author. He founded two successful high-tech companies (RadiSys and IP Fabrics), authored eight textbooks in the computer sciences, and made important contributions in microprocessor architecture. He holds a number of patents, including the original patent on "register scoreboarding" in microprocessor chips.[1] He has a BS in electrical engineering from Clarkson University, an MS in computer science from Syracuse University, and a PhD in computer science from the Polytechnic Institute of New York University. ] I got to read it early in my career, and applied it some, in commercial software projects I was a part of, or led, when I could. Very good book, IMO. There is a nice small testing-related question at the start of the book that many people don't answer well or fully.
  - pfdietz 1y ago
    As I recall this was a book that included the orthodoxy at the time that random testing was the worst kind of testing, to be avoided if possible. That turned out to be bullshit. Today, with computers many orders of magnitude faster, using randomly generated tests is a very cost effective away of testing, compared to carefully handcrafted tests. Use extremely cheap machine cycles to save increasingly expensive human time.
    - fuzztester 1y ago
      Interesting. Don't remember that from the book, but then, I read it long ago. I agree that random testing can be useful. For example, one kind of fuzzing is using tons of randomly generated test data against a program to try to find unexpected bugs. But I think both kinds have their place. Also, I think the author might have mean that random testing is bad when used with a small amount of test data, in which case I'd agree with him, because in that case, an equally small amount of carefully crafted test data would be the better option, e.g. using some test data in each equivalence class of the input.
marcosdumay 1y ago
When testing job candidates, sure, no doubt about that. For for learning, no, it's not. You should not spend as much time learning testing as you spend leaning data structures.
- Jtsummers 1y ago
  I feel like this mischaracterizes the blog. You seem to be taking this: > People should spend less time learning DSA, more time learning testing. And reading it as "More total time should be spent on learning testing than the total time spent learning DSA". That's one reading, another is that people are studying DSA too much, and testing too little. The ratio of total time can still be in favor of studying DSA more, but maybe instead of 10:1 it should be more like 8:1 or 5:1.
  - marcosdumay 1y ago
    That's a fair point. But then the author makes a blatant and unrealistic generalization about how much time people spend on each of those. Between CS undergrads and introduction to programming bootcamps, the variance on that number is extreme.
pshirshov 1y ago
Pure bullshit and incompetence. > esoteric things like Bloom filters, so you can find them later in the unlikely case you need them. They are not esoteric, they are trivial and extremely useful in many cases. > Less DSA, more testing. Testing can't cover all the cases by definition, why not property testing? Why not formal proofs? Plus, in our days, it's easy to delegate testcase writing to LLMs, while they literally cannot invent new useful AnDS.
- cogman10 1y ago
  > extremely useful in many cases. I've not ran into a case where I can apply a bloom filter. I keep looking because it always seems like it'd be useful. The problem I have is bloom filter has practically reverse characteristics from what I want. It gives false positives and true negatives. I most often want true positives and false negatives.
  - pshirshov 1y ago
    Assume that you need to build a large-scale search or analytics tool for example. All the sketch data structures (like cuckoo filters and especially hypermihashes) are extremely useful in these scenarios.
  - burch45 1y ago
    Its entire purpose is an optimization. You have an expensive operation. A bloom filter can tell you that you definitely don’t need to do that operation. So rather than wasting a lot of time unnecessarily doing that operation, you get the cheap Bloom filter Che most of the time and only occasionally have the false positive where you do the expensive thing when it turns out you didn’t need to. That as far as I am aware of is the only use case for a bloom filter. That said, I have used it for that purpose effectively several times in my career.
  - guiand 1y ago
    > true positives and false negatives That would be a simple cache in most instances.
- 1y ago
cogman10 1y ago
I agree. The main benefit of being familiar with how data structures and algorithms work is that you become familiar with their runtime characteristics and thus can know when to reach for them in a real problem. The author is correct here. You'll almost never need to implement a B-Tree. What's important is knowing that B-Trees have log n insertion times with good memory locality making them faster than simple binary trees. Knowing how the B-Tree works could help you in tuning it correctly, but otherwise just knowing the insertion/lookup efficiencies is enough.
hvb2 1y ago
This feels backwards. When you have a good understanding of data structures you have the luxury of testing. If you focus on testing over data structures, you might end up testing something that you didn't need to test because you used the wrong data structures. IMHO too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem
- jancsika 1y ago
  > IMHO too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem Not if the user can, say, farm 1,000,000 different rows 100 times over an hour and a half while gossiping with their office mates. I over Excel as Exhibit A.
- cogman10 1y ago
  That wasn't the thrust of the article. The article is saying that it's more important to write tests than it is to learn how to write data structures. It specifically says you should learn which data structures you should use, but don't focus on knowing how to implement all them. It calls out, specifically, that you should know that `sort` exists but you really don't need to know how to implement quicksort vs selection sort.
  - hvb2 1y ago
    No, it says learn data structures first, then focus on testing. You don't have to go super deep on all the sort algorithms, sure. That's like saying that learning testing implies writing a mocking library
    - MrJohz 1y ago
      I think the issue is that most DSA curriculums do go super deep on things like sort algorithms or linked lists or whatever else. Whereas testing is usually barely taught in universities or colleges, and when it is taught, it's usually very lightweight.
- 1y ago
nice_byte 1y ago
ignore this advice. spend plenty of time studying data structures and algorithms as well as computer architecture. these are actually difficult things that take a long time to understand and will have a positive impact on your career. study the underlying disciplines of your preferred domain. in general, focus on more fundamental things and limit the amount of time you spend on stupid shit like frameworks, build systems, quirks of an editor or a programming language. all these things will find a way to steal your time _anyway_, and your time is extremely precious. "testing" is not fundamental. there is no real skill to be learned there, it's just one of those things that will find a way to steal your time anyway so there is no point in focusing actively on it. put it that way: you will NEVER get the extra time to study fundamental theory. you will ALWAYS be forced to spend time to write tests. if you somehow find the time, spend it on things that are worth it.
- KevinMS 1y ago
  > "testing" is not fundamental. there is no real skill to be learned there, it's just one of those things that will find a way to steal your time anyway so there is no point in focusing actively on it. that's an edgy take and a red flag
  - nice_byte 1y ago
    it is not edgy whatsoever. it reflects the actual reality on the ground. nobody goes to school to learn how to use git or how to write unit tests. it's not something that needs to be actively "learned", you'll just absorb it eventually because you can't escape it. The more interesting and important things you will never "just absorb", you actually have to make a conscious effort to engage with them.
    - KevinMS 1y ago
      I'm replying to statements like this > "testing" is not fundamental. and > there is no real skill to be learned there one of the biggest problems that has plagued software is failed projects. There have been a lot of them, and its probably costs hundreds of billions of dollars. I can guarantee not one of those projects failed because somebody had to take the time to look up the best data structure. But I'll bet a lot of them failed because they didn't follow smart testing practices and collapsed under their own weight of complexity, untestability and inflexibility.
fastaguy88 1y ago
It really depends. Working on genome analysis, I once encountered/interrupted (by rebooting after a software update) a student who had been running an analysis for more than a week, because they had not pre-sorted the data. With pre-sorted data, it took a few minutes. Not everyone works on web sites using well-optimized libraries; some people need to know about N and Nlog(N) vs N^2.
- matheusmoreira 1y ago
  > some people need to know about N and Nlog(N) vs N^2. Every programmer should know enough to at least avoid accidentally making things quadratic. https://news.ycombinator.com/item?id=26296339 https://news.ycombinator.com/item?id=26296339
  - hetman 1y ago
    Indeed. As an anecdote, I've come across a self professed frontend UI guru writing quadratic code that worked fine in testing because it only had to display a few tens of items there, but at a complete loss why it was unusable in production.
  - pfdietz 1y ago
    Or, to recognize when they or someone else has done so and recover. It's often a case of "N won't be large here" and then later N does sometimes turn out to be large.
matheusmoreira 1y ago
> Of course some engineers need to implement hash tables, or sorting algorithms or whatever. > We love those engineers: they write libraries we can use off the shelf so we don’t have to implement them ourselves. The world needs to love "infrastructure developers" more. To me it seems only the killer app writing crowd is valued. Nobody really thinks about the work that goes into programming languages, libraries and tools. It's invisible work, taken for granted, often open source, not rarely unpaid. > It wasn’t opening a textbook to find the famous algorithm that would solve my problem. I had that exact experience. I'm working on my own programming language. After weeks of trying to figure something out by myself, someone told me to read Structure and Interpretation of Computer Programs. It literally had the exact algorithm I wanted.
- chamomeal 1y ago
  Dang I got this book a few weeks ago and still haven’t cracked it open. Maybe today is the day
  - alabhyajindal 1y ago
    This might be useful: https://docs.racket-lang.org/sicp-manual/index.html https://docs.racket-lang.org/sicp-manual/index.html
  - matheusmoreira 1y ago
    The algorithm I'm talking about is at the very end of the book. If you start reading it from start to finish you might stop before you reach it. Certainly happened to me. Someone had to point it out for me to realize SICP had the answer all along. https://eng.libretexts.org/Bookshelves/Computer_Science/Programming_and_Computation_Fundamentals/Structure_and_Interpretation_of_Computer_Programs_(Abelson_Sussman_and_Sussman)/05%3A_Computing_with_Register_Machines/5.04%3A_New_Page https://eng.libretexts.org/Bookshelves/Computer_Science/Prog... The explicit control evaluator. It's a register and stack machine which evaluates lisp expressions without transforming them into bytecode.
    - hinkley 1y ago
ChrisMarshallNY 1y ago
I agree with the article, but I'll bet a lot of others, don't. Discussions on Code Quality, don't fare well, here. Wouldn't surprise me, if the article already has flags. Of course, "testing," is in the eye of the beholder. Some folks are completely into TDD, and insist that you need to have 100% code coverage tests, before writing one line of application code, and some folks think that 100% code coverage unit tests, means that the system is fully tested. I've learned that it's a bit more nuanced than this[0]. [0] https://littlegreenviper.com/testing-harness-vs-unit/ https://littlegreenviper.com/testing-harness-vs-unit/
- general1465 1y ago
  Testing, especially vstest.console.exe in Visual Studio has carried my business really far. I have accumulated thousands of tests on my codebase usually based on customer requirements or on past bugs which I have been trying to replicate. I think that a lot of people dislike testing because a lot of tests can run for hours. In my case it is almost 6 hours from start to finish. However as a software developer I have accumulated a lot of computers which are kind of good and I don't want to throw them out yet but they are not really usable for current development - i.e. 8GB of RAM, 256GB SSD, i5 CPU from 2014 - That would be a punishment to use it with Visual Studio today. But it is a perfect machine for compiling in console i.e. dotnet build or msbuild and running tests via vstest glued together with PowerShell script. So this dedicated testing machine is running on changes over night and I will see if it passed or not and if not fix tests which did not passed. This setup may feel clunky, but it allows me to make sweeping changes in a codebase and be confident enough, that if the tests pass, it will very likely work for the customer too. The most obvious example where tests were carrying me around has been moving to .NET8 from .NET Framework 4.8. I have went from 90% failure rate on tests to all tests clear in like 3-4 iterations.
  - Izikiel43 1y ago
    You could have a pipeline in some cloud provider to run the tests, and distribute the load across machines if tests are independent to reduce the time, if that's more important. If it's ok to just run the overnight, keep it on.
Izikiel43 1y ago
Testing is the case I've found AI actually useful. Write one good test, maybe happy path, and then tell your AI to test for scenario XYZ and how it should fail, etc. The generated code is in general 90% there. This allows me to write many more tests than before to try to catch all scenarios.
glitchc 1y ago
The article fails to demonstrate how code-tests result in objectively better code. Many comp sci programs have courses on testing that cover TDD, unit testing and fuzzing, among other topics. Yet much of the safety critical code we rely on for critical infrastructure (nuclear reactors, aircraft, drones, etc) is not tested in-situ. It is tested via simulation, but there's minimal testing in the operating environment which can be quite complex. Instead the code follows carefully chosen design patterns, data structures and algorithms, to ensure that the code is hazard-free, fault-tolerant and capable of graceful degradation. So, testing has its place, but testing is really no better than simulation. And in simulation, the outputs are only as good as the inputs. It cannot guarantee code safety and is not a substitute for good software design (read: structures and algorithms). Having said that, fuzzing is a great way to find bugs in your code, and highly recommended for any software that exposes an API to other systems.
- azeirah 1y ago
  I don't understand what the difference between a simulation and a test is?
  - glitchc 1y ago
    There is none, and that's my point. Simulations themselves are contrived scenarios that are not representative of production environments.
  - qayxc 1y ago
    Mostly just semantics.
- MoreQARespect 1y ago
  >fails to demonstrate how code-tests result in objectively better code. Tests give the freedom to refactor which results in better code. >So, testing has its place, but testing is really no better than simulation Testing IS simulation and simulation IS testing. >And in simulation, the outputs are only as good as the inputs. It cannot guarantee code safety Only juniors think that you can get guarantees of code safety. Seniors look for ways to de-risk code, knowing that you're always trending towards a minima. One of the key skills in testing is defining good, realistic inputs.
burnt-resistor 1y ago
Sigh. Monochromatic myopia denying the need for holistic quality and mastery in multiple arenas and methodologies. Belts and suspenders, not just elastic waistbands.
- CyberDildonics 1y ago
  Oh if the holistic myopia is multiples of monochromatic does it really need elastic mastery? Sigh.
danielmarkbruce 1y ago
This will annoy a lot of folks, but: 1 - If you work on large scale software systems, especially infrastructure software of most types then you need to know and understand DSA and feel it in your bones. 2 - Most people work on crud apps or similar and don't really need to know this stuff. Many people in this camp don't realize that people working on 1 really do need to know this stuff. What someone says on this topic says more about what things they have worked on in their life than anything else.
- uncivilized 1y ago
  I already know the answer to this, but did you read the article? Ned addresses your concerns.
  - danielmarkbruce 1y ago
    No, he doesn't. He doesn't discuss the gigantic dividing line between the two different types of systems I categorize above. He also doesn't cover the "feel it in your bones" required in the type 1 systems. Spend a minute reading or listening to Jeff Dean talk, and you'll see what is required to build those types of systems. Spend some time somewhere working on those systems and you'll come across some folks who just have this ready to go and can apply it and the drop of a hat.
- ecshafer 1y ago
  > What someone says on this topic says more about what things they have worked on in their life than anything else. This is the crux of the debate. If you work on CRUD apps, you basically need to know hash maps, and lists, but getting better at SQL and writing clean code is good. But there are many areas where writing the right code vs the wrong code really matters. I was writing something the other day where one small in loop operation was the difference betweeen a method running in miliseconds and minutes. Or choose the right data structure can simplify a feature into 1/10th the code and makes it run 100x better than the wrong one.
  - MoreQARespect 1y ago
karmakaze 1y ago
The context what you should spend time to learn starting out. TL;DR > Here is what I think in-the-trenches software engineers should know about data structures and algorithms: [...] > If you want to prepare yourself for a career, and also stand out in job interviews, learn how to write tests: [...] I feel like I keep writing these little context comments to fix the problem of clickbait titles or those lacking context. It helps to frame the rest of the comments which might be coming at it from different angles.
atmavatar 1y ago
The title is unfortunately more than a little irresponsible, considering it's the norm for many (most?) to read only the title. There is no dichotomy here: you need to know testing as well as data structures and algorithms. However, the thrust of the article itself I largely agree with -- that it's less important to have such in-depth knowledge about data structures and algorithms that you can implement them from scratch and from memory. Nearly any modern language you'll program in includes a standard library robust enough that you'll almost never have to implement many of the most well-known data structures and algorithms yourself. The caveat: you still need to know enough about how they work to be capable of selecting which to use. In the off-chance you do have to implement something yourself, there's no shortage of reference material available.
jerf 1y ago
This is one of the things I'd tune in the current curriculum. When I went to college in the late 1990s, we were right on the verge of a major transition to DSAs being something every programmer would implement themselves to something that you just pick up out of your libraries. So it makes sense that we would have some pretty heavy-duty labs on implementing very basic data structures. That said, I escaped into the dynamic programming world for the next 15 years or so, so I almost never actually did anything of significance with this. And now even in the static world, I almost never do anything with this stuff directly because it's all libraries for them now too. Even a lot of modern data structures work is just using associative maps and arrays together properly. So I would agree that we could A: spend somewhat less time on this in the curriculum and B: tune it to more about how to use arrays and maps and less about how to bit bang efficient hash tables. People always get frosty about trying to remove or even "tune down" the amount of time spent in a curriculum, but consider the number of things you want to add and consider that curricula are essentially zero-sum games; you can't add to them without removing something. If we phrase this in terms of "what else could we be teaching other than a fifth week on pointer-based data structures" I imagine it'll sound less horrifying to tweak this. Not that it'll be tweaked, of course. But it'd be nice to imagine that I could live in a world where we could have reasonable discussions about what should be in them.
- jkhdigital 1y ago
  About 20 years ago I failed out of the undergrad CS program at UIUC because I thought I was smart enough to skip most lectures. I did manage to get an A in the C++ Data Structures course because the lectures were recorded and I just binged them all the night before each test. Anyways, now I’m a full-time lecturer teaching undergraduate CS courses (long story) and I’m actually shaping curriculum. As soon as I read this article I thought “I need to tell my data structures students to read this” because it echoes a lot of what I’ve been saying in class. Case in point: right after two lectures covering the ArrayList versus LinkedList implementations of the Java List interface, I spent an entire lecture on JUnit and live-coded a performance test suite that produced actual data to back up our discussions of big-O complexity. The best part of all? They learned about JIT compilation in the JVM firsthand because it completely blew apart the expected test results.
varjag 1y ago
Sudokugate flashbacks.
zeroCalories 1y ago
The reason you study ds&a is because it's a difficult skill. Testing is incredibly straightforward and easy to learn. The reason tests suck is because the infra suck, the eng is lazy, or both.
JackSlateur 1y ago
I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all. You do your test, all good, you can roll in production ! But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything about that, and "your tests were green". Algorithms are the backbone tools of programming. Knowing them help us, ignoring them burdens us.
- wubrr 1y ago
  I mean, in your example you just have an incomplete test suite. (Though writing a complete one is often unrealistic) While understanding algorithms and data structures is important, the only way you really know how well it works, and how well it's implemented is by thoroughly testing it. There are an infinite amount of clever algorithms out there with terrible implementations. You need both.
  - JackSlateur 1y ago
    Testing concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;
    - wubrr 1y ago
      > Testing concurrency is extremely hard Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder. > I truly wonder if there is real world tests around this Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and the systems involved is required to write a proper test suite. > For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Take those queries and run them 1000x+ times concurrently in a loop. That will catch most common issues. If you want to go a step further you can build a setup to execute your queries in any desired order.
jsd1982 1y ago
Are we assuming that "testing" is limited to only exercising the single-threaded behavior of a function? I'm curious how others approach effective testing of multi-threaded behavior.
IvyMike 1y ago
Always gonna have to side with Peter Norvig on this one: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-... > They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bunch of tests.” But then he never got anywhere. He had five different blog posts and in each one he wrote a little bit more and wrote lots of tests but he never got anything working because he didn’t know how to solve the problem. I actually knew—from AI—that, well, there’s this field of constraint propagation—I know how that works. There’s this field of recursive search—I know how that works. And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases.
- IvyMike 1y ago
  More context, from an earlier HN comment: https://news.ycombinator.com/item?id=3033446 https://news.ycombinator.com/item?id=3033446
- pncnmnp 1y ago
  I love what Norvig said. I can relate to it. As far as data structures are concerned, I think it's worth playing smart with your tests - focus on the "invariants" and ensure their integrity. A classic example of invariant I can think of is the min-heap - node N is less than or equal to the value of its children - the heap property. Five years from now, you might forget the operations and the nuanced design principles, but the invariants might stay well in your memory.
- roxolotl 1y ago
  This totally misses the point of the article. The article agrees that knowing when a problem is a data structure and algo problem is a key strength. The article also isn’t saying that all development should be done TDD. The point of the article is that knowing how to test well is more useful than memorizing solutions to algo problems. You can always look those up.
29athrowaway 1y ago
Learning both is not mutually exclusive.
sakesun 1y ago
Just realise that I have been reading his blog for two decades already.
- rossant 1y ago
  Same. Love his blog.
JoeAltmaier 1y ago
We wrote a conferencing app and server (years before Zoom). Tested the server by having automated headless apps run in gangs, a hundred at a time, hopping from conversation to conversation, turning mic and camera on and off, logging out and logging back in. Used it for years, the Bot Army we called it. Responsible for our rock-solid quality reputation. Not API design or test classes or constraints or anything. Just, trying the damn thing, in large cases, for a long time. When it ran an hour, we celebrated. When it ran overnight, we celebrated. When it ran a week we celebrated, and called that good enough.
- rossant 1y ago
  How much work was it to go from 1 hour to 1 week? How many issues have you discovered, what were they? Genuinely interested.
  - JoeAltmaier 1y ago
    It took a big fraction of our energy to get to each new stage. Always it was something new and unexpected. It was quite a while ago, but lets see what I remember. Some good fraction were mismatches between the app (bot) state and the server state. A bot would be expecting a message and stall. The server thought it had said enough. The app side used a lot of libraries, which it turns out are never as robust as advertised. They leak, race, are very particular about call order. Have no sense of humor if they're still connecting and a disconnect call is made, for instance. The open source server components were fragile. In one instance, the database consistency library had an update where, for performance, a success message was returned before the operation upstream was complete. Which broke, utterly, the consistency promise that was the entire point of using that product. A popular message library created a timer on each instantiation. Cancelled it, but in typical Java fashion didn't unlink it. So, leak. Tiny, but you do it enough times, even the biggest server instance runs out of memory. We ran bots on Windows, Linux, even a Mac. Their network libraries had wildly different socket support. We'd run out of sockets! They got garbage collected after a time, but the timer could be enormous (minutes). Our server used a message-distribution component to 'shard' messages. It had a hard limit on message dispatching per second. I had to aggregate the client app messages (we used UDP and a proprietary signaling protocol) to drop the message rate (ethernet packet rate) by an order of magnitude. Added a millisecond of latency, which was actually important and another problem. Add the usual Java null pointers, order-dependent service termination rules (never documented), object lifetime surprises. It went on and on. Each doubling of survival-time the issues got more arcane and more interesting. Sometimes took a new tool or technique to ferret out the problem. To be honest, I was in hog heaven. Kept my brain plastic for a long time.
OfflineSergio 1y ago
I think they enough unnecessary content in universities that there be room for both DSA and testing.
rr808 1y ago
Do the people who love testing run JS or Python or something that compiles really quick? I've worked on some big projects and just to make a change, compile and run a unit test often takes 5-15 minutes. TDD works great if you have a trivial library in an interpreted language.
- yakshaving_jgt 1y ago
  Understanding how to write tests in an economical fashion is a skill in itself. With care, you can write fast (and effective) tests in Haskell (and, I’m sure, every language ever).
- markmark 1y ago
  Testing doesn't imply TDD.
ngcc_hk 1y ago
Can a program function without data, algorithm or testing or users or programmers. Now which one is better … sorry what rubbish question is that. All are important. And limited time you still have to consider all these … noth8ng is better.
quotemstr 1y ago
> I see new learners asking about “DSA” a lot. I've noticed this "DSA" acronym appearing overnight. I can't recall people using it this much (at all actually) even six months ago. Where did it come from? Why do we suddenly need a term to talk about the concept?
- burch45 1y ago
  That is the standard acronym for the course in American universities and has been for many decades.
  - quotemstr 1y ago
    Maybe so, but that doesn't explain why have people suddenly started using it more.
curtisszmania 1y ago
[dead]
lolive 1y ago
Data structures are overrated. One day, a trainee told me that maybe a Set would be better than an array in one of my codes. I fired this ignorant immediately. #noOverengineering #arrayIsGoodEnough
teo_zero 1y ago
As usual, the title doesn't reflect the message of the article. While critical against learning tons of algorithms and structures by heart, the author advocates understanding what they do and why you would want to use them. This is what it means to master TSA. The last part denounces that testing is not tought enough, and learning it may be beneficial for your career. It's never said that testing is better than TSA, just that we would need more.
Arun2009 1y ago
I have always thought of DSA as a proxy for a subset of general software development skills: the ability to translate a problem into computer science or programming terms, implement it in code, and argue that the implementation is correct and efficient. Skill in solving DSA problems can signal both an aptitude for absorbing computer science knowledge in general and a capacity for solving problems through programming. It's not the whole thing, but it's certainly an important component. It’s not unlike a research mathematician being expected to solve quadratic equations. He may not need them in his day-to-day work, but with a little preparation he should be able to handle them. If he struggles with quadratic equations in an interview where such knowledge is expected, that would raise a red flag about his training.
- solumos 1y ago
  > It’s not unlike a research mathematician being expected to solve quadratic equations. He may not need them in his day-to-day work, but with a little preparation he should be able to handle them. If he struggles with quadratic equations in an interview where such knowledge is expected, that would raise a red flag about his training. The absurdity of this indicates a big part of what’s wrong with modern SWE interviews. You wouldn’t ask a research mathematician to derive the quadratic formula, and you wouldn’t give a writer a spelling test — passing or not isn’t related to their aptitude or proficiency. Additionally, imagine interviewing a complex analyst and quizzing them only on calc 1 — sure, it may be a course that they were expected to take at some point, but it really has little to do with what their actual work entails, and so half of the interview they’d be trying to slot what the proper layer of abstraction is in the limited context of the problem.
- pezgrande 1y ago
  Maybe in some companies. At "Enterprise" level your control-alt-f skills are more valuable (for your average IC) based on my experience.
jillesvangurp 1y ago
Well, you need both. You are going to suck at implementing algorithms if you don't learn how to test they actually work as intended. Most algorithms are used in library form. Unless you are writing those libraries, you probably should not be reinventing a lot of wheels. So there's a valid argument there that most of the stuff you learn as part of your computer science courses, you will not be implementing over and over again (if ever). So, you could argue that testing is a more universal skill that you need either way. But algorithms can come up once in a while. And it helps if you can guesstimate complexity of various algorithms and make some trade offs over picking one or the other. The skill you learn in college is not any particular algorithm but a broad knowledge of which mainstream ones are there, how they work, their tradeoffs, and the skill of implementing those or similar algorithms. You gain the skill of good judgment, being able to figure out how stuff works, and general intuition of how things are done at a high level. I've never developed a file system. But I know tree data structures such as b trees and red black trees have something to do with it. It's been decades since I looked at that stuff. But I could read up in an afternoon or so if it comes up. That doesn't qualify to start working on a file system. But, I don't think that's going to come up anyway. I have plenty of other things to do. I do dabble a bit with search algorithms once in a while. More of a hobby than work related. But there's some room in that space for being able to do some basic things with algorithms instead of using some prefab search product.
mekoka 1y ago
Many programmers think that the way DS&A work is that one beautiful spring morning, some requirements fall onto your lap where the words "nodes" and "edges" are neatly circled, thus marking a key moment in your life where you can finally put to use that old dusty algorithm book. Meanwhile, trees and graphs have been flying up to your face, slapping you left and right, every single day of your career and you kept repeating to yourself "see, you don't need all that DS&A stuff". I think the article is unnecessarily trying to contrast two orthogonal and separately useful software development concerns. DS&A are about having the most efficient tools for the specific job. Testing is about ensuring that results match expectations. But I'd be surprised by anyone who only learns DS&A in theory and also naturally develops the related, but subtler skill of recognizing classes of problems that match up to particular algorithms. It's an almost tacit skill, which many programmers don't have despite knowing some of the classic DS&A, because it requires to linger a bit longer in the material. And that's the real gap leetcode type challenges try to bridge. The more you do them, the more you reinforce an intuitive understanding of involved structures and techniques, including some of the subtler properties and oft undocumented corner cases. What looks like "memorization" to some is actually more of an almost indelible grokking, which you carry with you in professional programming. Admittedly, people who only do leetcode type challenges and never write software in the real world don't know which parts of these challenges are useless (in my opinion, the puzzles and lateral thinking parts). But people who never do these challenges also know squat about how useful they can be at honing one's "real life" problem solving and coding skills.
- specialist 1y ago
  > ...unnecessarily trying to contrast two orthogonal and separately useful software development concerns. > ...only do leetcode type challenges and never write software in the real world... Respectfully, I agree with OC's point about testing vs implementation. (Both are programming.) It's been a (long) while since I've had peers who regularly, usefully tested their own work. Like this OC, I too have struggled to articulate why leetcode has tiny IRL relevance. How's this: A useful distinction may be implementation vs usage. Of course, a tiny handful of people are entrusted with implementing libraries of algorithms, so should be properly vetted for that work. Ditto crypto, parsing, and other misc arcane arts. OC's point remains that leetcode hazing ignores the majority of actual work as a programmer. Such as fixing bugs, testing, modeling, documenting, enduring meetings, etc. -- IIRC, I haven't implemented a sort algorithm, either in anger or for fun, since my BASIC & Pascal days. Why would I? But I have recast real world problems as a Traveling Sales Person problems a handful of times. As you well know, modeling using well known data structures, to enable using stock algorithms, is a big part of the job. I'd rather interviews verified applicable skills. Such as data modeling, sequence diagrams, etc. And maybe some lightweight arch like (from the hip) caching, serialization, indexing, queuing and backpressure, locks, and validation (and whatever other bog standard stuff is immediately relevant). -- (Oops: I did recently implement topo sort, just to understand it better, even though I was using a ready-baked solution. Just like +25 years ago I implemented (poorly) taboo, simulated annealing, etc. during my brief optimization kick.)
lunias 1y ago
"If my Grandmother had wheels, she would have been a bike."
gdubs 1y ago
I find that test-driven advocates border on a religious way of thinking about software development, and I also find it sucks joy out of coding. Does that mean I don't think tests are valuable – no, that's not what I'm saying. There are critical pieces of code where testing will bring you more joy because you'll catch very bad bugs that would have really ruined your day. But as a way of constructing an application, it just has always felt really dull and uninspiring. I know people will vehemently disagree with this. But, sometimes you gotta write the novel once to know what story you're telling. Then you can go back, edit, put tests in the critical places.
BrenBarn 1y ago
This is a dangerous statement, although I don't think that was the author's intent: > Before I get to other opinions, let me make clear that anything you can do to help you get a job is a good thing to do. There are lots and lots of very evil things that will help you get a job, especially with companies that want to do evil things. Those are definitely not good things to do. I think the author intended a more restricted context like "any skill you can learn that will help you get a job is a good skill to learn from a technical perspective" (i.e., it might still be ethically wrong). This may seem pedantic but I've known some young graduates recently who went to work for very evil companies, and I think it's easy for impressionable minds to read statements like this and think "Yeah, see, everyone does it, who cares" without filling in the ethical background that the author assumes.

188 points by rsyring 1y ago

wjrb 1y ago

Are there any resources out there that anyone can recommend for learning testing in the way the author describes? In-the-trenches experience (especially "good" or "doing it right" experience) can be hard to come by; and why not stand on the shoulders of giants when learning it the first time?

cogman10 1y ago
Resources, none that I'm aware of. I generally think this is an OK way to look at testing [1], though I think it goes too far if you completely adopt their framework. The boil down the tests I like to see. Structure them with "Given/when/then" statements. You don't need a framework for this, just make method calls with whatever unit test framework you are using. Keep the methods small, don't do a whole lot of "then"s, split that into multiple tests. Structure your code so that you aren't testing too deep. Ideally, you don't need to stand up your entire environment to run a test. But do write some of those tests, they are important for catching issues that can hide between unit tests. [1] https://cucumber.io/docs/bdd/ https://cucumber.io/docs/bdd/
Jtsummers 1y ago
Working Effectively with Legacy Code by Michael Feathers. It spends a lot of time on how to introduce testability into existing software systems that were not designed for testing. Property-Based Testing with PropEr, Erlang, and Elixir by Fred Hebert. While a book about a particular tool (PropEr) and pair of languages (Erlang and Elixir), it's a solid introduction to property-based testing. The techniques described transfer well to other PBT systems and other languages. Test-Driven Development by Kent Beck. https://www.fuzzingbook.org/ https://www.fuzzingbook.org/ by Zeller et al. and https://www.debuggingbook.org/ https://www.debuggingbook.org/ by Andreas Zeller. The latter is technically about debugging, but it has some specific techniques that you can incorporate into how you test software. Like Delta Debugging, also described in a paper by Zeller et al. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=988498 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=988.... I'm not sure of other books I can recommend, the rest I know is from learning on the job or studying specific tooling and techniques.
- GrumpyYoungMan 1y ago
  TDD is a development methodology, not a testing methodology. The main thing it does is check whether the developer implemented what they thought they should be implementing, which is not necessarily what the spec actually says to implement or what the end user expects.
  - Jtsummers 1y ago
    It's still a useful technique and way to apply testing to development. But yes, it's not the best resource in telling you what tests to write, more about how they can be applied effectively. Which is a skill that seems absent in many professionals.
fuzztester 1y ago
The Art of Software Testing. New York: Wiley, 1979 The Art of Software Testing, Second Edition. with Tom Badgett and Todd M. Thomas, New York: Wiley, 2004. It is by Glenford Myers (and others). https://en.m.wikipedia.org/wiki/Glenford_Myers https://en.m.wikipedia.org/wiki/Glenford_Myers From the top of that page: [ Glenford Myers (born December 12, 1946) is an American computer scientist, entrepreneur, and author. He founded two successful high-tech companies (RadiSys and IP Fabrics), authored eight textbooks in the computer sciences, and made important contributions in microprocessor architecture. He holds a number of patents, including the original patent on "register scoreboarding" in microprocessor chips.[1] He has a BS in electrical engineering from Clarkson University, an MS in computer science from Syracuse University, and a PhD in computer science from the Polytechnic Institute of New York University. ] I got to read it early in my career, and applied it some, in commercial software projects I was a part of, or led, when I could. Very good book, IMO. There is a nice small testing-related question at the start of the book that many people don't answer well or fully.
- pfdietz 1y ago
  As I recall this was a book that included the orthodoxy at the time that random testing was the worst kind of testing, to be avoided if possible. That turned out to be bullshit. Today, with computers many orders of magnitude faster, using randomly generated tests is a very cost effective away of testing, compared to carefully handcrafted tests. Use extremely cheap machine cycles to save increasingly expensive human time.
  - fuzztester 1y ago
    Interesting. Don't remember that from the book, but then, I read it long ago. I agree that random testing can be useful. For example, one kind of fuzzing is using tons of randomly generated test data against a program to try to find unexpected bugs. But I think both kinds have their place. Also, I think the author might have mean that random testing is bad when used with a small amount of test data, in which case I'd agree with him, because in that case, an equally small amount of carefully crafted test data would be the better option, e.g. using some test data in each equivalence class of the input.

marcosdumay 1y ago

When testing job candidates, sure, no doubt about that. For for learning, no, it's not. You should not spend as much time learning testing as you spend leaning data structures.

Jtsummers 1y ago
I feel like this mischaracterizes the blog. You seem to be taking this: > People should spend less time learning DSA, more time learning testing. And reading it as "More total time should be spent on learning testing than the total time spent learning DSA". That's one reading, another is that people are studying DSA too much, and testing too little. The ratio of total time can still be in favor of studying DSA more, but maybe instead of 10:1 it should be more like 8:1 or 5:1.
- marcosdumay 1y ago
  That's a fair point. But then the author makes a blatant and unrealistic generalization about how much time people spend on each of those. Between CS undergrads and introduction to programming bootcamps, the variance on that number is extreme.

pshirshov 1y ago

Pure bullshit and incompetence. > esoteric things like Bloom filters, so you can find them later in the unlikely case you need them. They are not esoteric, they are trivial and extremely useful in many cases. > Less DSA, more testing. Testing can't cover all the cases by definition, why not property testing? Why not formal proofs? Plus, in our days, it's easy to delegate testcase writing to LLMs, while they literally cannot invent new useful AnDS.

cogman10 1y ago
> extremely useful in many cases. I've not ran into a case where I can apply a bloom filter. I keep looking because it always seems like it'd be useful. The problem I have is bloom filter has practically reverse characteristics from what I want. It gives false positives and true negatives. I most often want true positives and false negatives.
- pshirshov 1y ago
  Assume that you need to build a large-scale search or analytics tool for example. All the sketch data structures (like cuckoo filters and especially hypermihashes) are extremely useful in these scenarios.
- burch45 1y ago
  Its entire purpose is an optimization. You have an expensive operation. A bloom filter can tell you that you definitely don’t need to do that operation. So rather than wasting a lot of time unnecessarily doing that operation, you get the cheap Bloom filter Che most of the time and only occasionally have the false positive where you do the expensive thing when it turns out you didn’t need to. That as far as I am aware of is the only use case for a bloom filter. That said, I have used it for that purpose effectively several times in my career.
- guiand 1y ago
  > true positives and false negatives That would be a simple cache in most instances.
1y ago

cogman10 1y ago

I agree. The main benefit of being familiar with how data structures and algorithms work is that you become familiar with their runtime characteristics and thus can know when to reach for them in a real problem. The author is correct here. You'll almost never need to implement a B-Tree. What's important is knowing that B-Trees have log n insertion times with good memory locality making them faster than simple binary trees. Knowing how the B-Tree works could help you in tuning it correctly, but otherwise just knowing the insertion/lookup efficiencies is enough.

hvb2 1y ago

This feels backwards. When you have a good understanding of data structures you have the luxury of testing. If you focus on testing over data structures, you might end up testing something that you didn't need to test because you used the wrong data structures. IMHO too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem

jancsika 1y ago
> IMHO too often people dont consider big O because it works fine with their 10 row test case.... And then it grinds to a halt when given a real problem Not if the user can, say, farm 1,000,000 different rows 100 times over an hour and a half while gossiping with their office mates. I over Excel as Exhibit A.
cogman10 1y ago
That wasn't the thrust of the article. The article is saying that it's more important to write tests than it is to learn how to write data structures. It specifically says you should learn which data structures you should use, but don't focus on knowing how to implement all them. It calls out, specifically, that you should know that `sort` exists but you really don't need to know how to implement quicksort vs selection sort.
- hvb2 1y ago
  No, it says learn data structures first, then focus on testing. You don't have to go super deep on all the sort algorithms, sure. That's like saying that learning testing implies writing a mocking library
  - MrJohz 1y ago
    I think the issue is that most DSA curriculums do go super deep on things like sort algorithms or linked lists or whatever else. Whereas testing is usually barely taught in universities or colleges, and when it is taught, it's usually very lightweight.
1y ago

nice_byte 1y ago

ignore this advice. spend plenty of time studying data structures and algorithms as well as computer architecture. these are actually difficult things that take a long time to understand and will have a positive impact on your career. study the underlying disciplines of your preferred domain. in general, focus on more fundamental things and limit the amount of time you spend on stupid shit like frameworks, build systems, quirks of an editor or a programming language. all these things will find a way to steal your time _anyway_, and your time is extremely precious. "testing" is not fundamental. there is no real skill to be learned there, it's just one of those things that will find a way to steal your time anyway so there is no point in focusing actively on it. put it that way: you will NEVER get the extra time to study fundamental theory. you will ALWAYS be forced to spend time to write tests. if you somehow find the time, spend it on things that are worth it.

KevinMS 1y ago
> "testing" is not fundamental. there is no real skill to be learned there, it's just one of those things that will find a way to steal your time anyway so there is no point in focusing actively on it. that's an edgy take and a red flag
- nice_byte 1y ago
  it is not edgy whatsoever. it reflects the actual reality on the ground. nobody goes to school to learn how to use git or how to write unit tests. it's not something that needs to be actively "learned", you'll just absorb it eventually because you can't escape it. The more interesting and important things you will never "just absorb", you actually have to make a conscious effort to engage with them.
  - KevinMS 1y ago
    I'm replying to statements like this > "testing" is not fundamental. and > there is no real skill to be learned there one of the biggest problems that has plagued software is failed projects. There have been a lot of them, and its probably costs hundreds of billions of dollars. I can guarantee not one of those projects failed because somebody had to take the time to look up the best data structure. But I'll bet a lot of them failed because they didn't follow smart testing practices and collapsed under their own weight of complexity, untestability and inflexibility.

fastaguy88 1y ago

It really depends. Working on genome analysis, I once encountered/interrupted (by rebooting after a software update) a student who had been running an analysis for more than a week, because they had not pre-sorted the data. With pre-sorted data, it took a few minutes. Not everyone works on web sites using well-optimized libraries; some people need to know about N and Nlog(N) vs N^2.

matheusmoreira 1y ago
> some people need to know about N and Nlog(N) vs N^2. Every programmer should know enough to at least avoid accidentally making things quadratic. https://news.ycombinator.com/item?id=26296339 https://news.ycombinator.com/item?id=26296339
- hetman 1y ago
  Indeed. As an anecdote, I've come across a self professed frontend UI guru writing quadratic code that worked fine in testing because it only had to display a few tens of items there, but at a complete loss why it was unusable in production.
- pfdietz 1y ago
  Or, to recognize when they or someone else has done so and recover. It's often a case of "N won't be large here" and then later N does sometimes turn out to be large.

matheusmoreira 1y ago

> Of course some engineers need to implement hash tables, or sorting algorithms or whatever. > We love those engineers: they write libraries we can use off the shelf so we don’t have to implement them ourselves. The world needs to love "infrastructure developers" more. To me it seems only the killer app writing crowd is valued. Nobody really thinks about the work that goes into programming languages, libraries and tools. It's invisible work, taken for granted, often open source, not rarely unpaid. > It wasn’t opening a textbook to find the famous algorithm that would solve my problem. I had that exact experience. I'm working on my own programming language. After weeks of trying to figure something out by myself, someone told me to read Structure and Interpretation of Computer Programs. It literally had the exact algorithm I wanted.

chamomeal 1y ago
Dang I got this book a few weeks ago and still haven’t cracked it open. Maybe today is the day
- alabhyajindal 1y ago
  This might be useful: https://docs.racket-lang.org/sicp-manual/index.html https://docs.racket-lang.org/sicp-manual/index.html
- matheusmoreira 1y ago
  The algorithm I'm talking about is at the very end of the book. If you start reading it from start to finish you might stop before you reach it. Certainly happened to me. Someone had to point it out for me to realize SICP had the answer all along. https://eng.libretexts.org/Bookshelves/Computer_Science/Programming_and_Computation_Fundamentals/Structure_and_Interpretation_of_Computer_Programs_(Abelson_Sussman_and_Sussman)/05%3A_Computing_with_Register_Machines/5.04%3A_New_Page https://eng.libretexts.org/Bookshelves/Computer_Science/Prog... The explicit control evaluator. It's a register and stack machine which evaluates lisp expressions without transforming them into bytecode.
  - hinkley 1y ago

ChrisMarshallNY 1y ago

I agree with the article, but I'll bet a lot of others, don't. Discussions on Code Quality, don't fare well, here. Wouldn't surprise me, if the article already has flags. Of course, "testing," is in the eye of the beholder. Some folks are completely into TDD, and insist that you need to have 100% code coverage tests, before writing one line of application code, and some folks think that 100% code coverage unit tests, means that the system is fully tested. I've learned that it's a bit more nuanced than this[0]. [0] https://littlegreenviper.com/testing-harness-vs-unit/ https://littlegreenviper.com/testing-harness-vs-unit/

general1465 1y ago
Testing, especially vstest.console.exe in Visual Studio has carried my business really far. I have accumulated thousands of tests on my codebase usually based on customer requirements or on past bugs which I have been trying to replicate. I think that a lot of people dislike testing because a lot of tests can run for hours. In my case it is almost 6 hours from start to finish. However as a software developer I have accumulated a lot of computers which are kind of good and I don't want to throw them out yet but they are not really usable for current development - i.e. 8GB of RAM, 256GB SSD, i5 CPU from 2014 - That would be a punishment to use it with Visual Studio today. But it is a perfect machine for compiling in console i.e. dotnet build or msbuild and running tests via vstest glued together with PowerShell script. So this dedicated testing machine is running on changes over night and I will see if it passed or not and if not fix tests which did not passed. This setup may feel clunky, but it allows me to make sweeping changes in a codebase and be confident enough, that if the tests pass, it will very likely work for the customer too. The most obvious example where tests were carrying me around has been moving to .NET8 from .NET Framework 4.8. I have went from 90% failure rate on tests to all tests clear in like 3-4 iterations.
- Izikiel43 1y ago
  You could have a pipeline in some cloud provider to run the tests, and distribute the load across machines if tests are independent to reduce the time, if that's more important. If it's ok to just run the overnight, keep it on.

Izikiel43 1y ago

Testing is the case I've found AI actually useful. Write one good test, maybe happy path, and then tell your AI to test for scenario XYZ and how it should fail, etc. The generated code is in general 90% there. This allows me to write many more tests than before to try to catch all scenarios.

glitchc 1y ago

The article fails to demonstrate how code-tests result in objectively better code. Many comp sci programs have courses on testing that cover TDD, unit testing and fuzzing, among other topics. Yet much of the safety critical code we rely on for critical infrastructure (nuclear reactors, aircraft, drones, etc) is not tested in-situ. It is tested via simulation, but there's minimal testing in the operating environment which can be quite complex. Instead the code follows carefully chosen design patterns, data structures and algorithms, to ensure that the code is hazard-free, fault-tolerant and capable of graceful degradation. So, testing has its place, but testing is really no better than simulation. And in simulation, the outputs are only as good as the inputs. It cannot guarantee code safety and is not a substitute for good software design (read: structures and algorithms). Having said that, fuzzing is a great way to find bugs in your code, and highly recommended for any software that exposes an API to other systems.

azeirah 1y ago
I don't understand what the difference between a simulation and a test is?
- glitchc 1y ago
  There is none, and that's my point. Simulations themselves are contrived scenarios that are not representative of production environments.
- qayxc 1y ago
  Mostly just semantics.
MoreQARespect 1y ago
>fails to demonstrate how code-tests result in objectively better code. Tests give the freedom to refactor which results in better code. >So, testing has its place, but testing is really no better than simulation Testing IS simulation and simulation IS testing. >And in simulation, the outputs are only as good as the inputs. It cannot guarantee code safety Only juniors think that you can get guarantees of code safety. Seniors look for ways to de-risk code, knowing that you're always trending towards a minima. One of the key skills in testing is defining good, realistic inputs.

burnt-resistor 1y ago

Sigh. Monochromatic myopia denying the need for holistic quality and mastery in multiple arenas and methodologies. Belts and suspenders, not just elastic waistbands.

CyberDildonics 1y ago
Oh if the holistic myopia is multiples of monochromatic does it really need elastic mastery? Sigh.

danielmarkbruce 1y ago

This will annoy a lot of folks, but: 1 - If you work on large scale software systems, especially infrastructure software of most types then you need to know and understand DSA and feel it in your bones. 2 - Most people work on crud apps or similar and don't really need to know this stuff. Many people in this camp don't realize that people working on 1 really do need to know this stuff. What someone says on this topic says more about what things they have worked on in their life than anything else.

uncivilized 1y ago
I already know the answer to this, but did you read the article? Ned addresses your concerns.
- danielmarkbruce 1y ago
  No, he doesn't. He doesn't discuss the gigantic dividing line between the two different types of systems I categorize above. He also doesn't cover the "feel it in your bones" required in the type 1 systems. Spend a minute reading or listening to Jeff Dean talk, and you'll see what is required to build those types of systems. Spend some time somewhere working on those systems and you'll come across some folks who just have this ready to go and can apply it and the drop of a hat.
ecshafer 1y ago
> What someone says on this topic says more about what things they have worked on in their life than anything else. This is the crux of the debate. If you work on CRUD apps, you basically need to know hash maps, and lists, but getting better at SQL and writing clean code is good. But there are many areas where writing the right code vs the wrong code really matters. I was writing something the other day where one small in loop operation was the difference betweeen a method running in miliseconds and minutes. Or choose the right data structure can simplify a feature into 1/10th the code and makes it run 100x better than the wrong one.
- MoreQARespect 1y ago

karmakaze 1y ago

The context what you should spend time to learn starting out. TL;DR > Here is what I think in-the-trenches software engineers should know about data structures and algorithms: [...] > If you want to prepare yourself for a career, and also stand out in job interviews, learn how to write tests: [...] I feel like I keep writing these little context comments to fix the problem of clickbait titles or those lacking context. It helps to frame the rest of the comments which might be coming at it from different angles.

atmavatar 1y ago

The title is unfortunately more than a little irresponsible, considering it's the norm for many (most?) to read only the title. There is no dichotomy here: you need to know testing as well as data structures and algorithms. However, the thrust of the article itself I largely agree with -- that it's less important to have such in-depth knowledge about data structures and algorithms that you can implement them from scratch and from memory. Nearly any modern language you'll program in includes a standard library robust enough that you'll almost never have to implement many of the most well-known data structures and algorithms yourself. The caveat: you still need to know enough about how they work to be capable of selecting which to use. In the off-chance you do have to implement something yourself, there's no shortage of reference material available.

jerf 1y ago

This is one of the things I'd tune in the current curriculum. When I went to college in the late 1990s, we were right on the verge of a major transition to DSAs being something every programmer would implement themselves to something that you just pick up out of your libraries. So it makes sense that we would have some pretty heavy-duty labs on implementing very basic data structures. That said, I escaped into the dynamic programming world for the next 15 years or so, so I almost never actually did anything of significance with this. And now even in the static world, I almost never do anything with this stuff directly because it's all libraries for them now too. Even a lot of modern data structures work is just using associative maps and arrays together properly. So I would agree that we could A: spend somewhat less time on this in the curriculum and B: tune it to more about how to use arrays and maps and less about how to bit bang efficient hash tables. People always get frosty about trying to remove or even "tune down" the amount of time spent in a curriculum, but consider the number of things you want to add and consider that curricula are essentially zero-sum games; you can't add to them without removing something. If we phrase this in terms of "what else could we be teaching other than a fifth week on pointer-based data structures" I imagine it'll sound less horrifying to tweak this. Not that it'll be tweaked, of course. But it'd be nice to imagine that I could live in a world where we could have reasonable discussions about what should be in them.

jkhdigital 1y ago
About 20 years ago I failed out of the undergrad CS program at UIUC because I thought I was smart enough to skip most lectures. I did manage to get an A in the C++ Data Structures course because the lectures were recorded and I just binged them all the night before each test. Anyways, now I’m a full-time lecturer teaching undergraduate CS courses (long story) and I’m actually shaping curriculum. As soon as I read this article I thought “I need to tell my data structures students to read this” because it echoes a lot of what I’ve been saying in class. Case in point: right after two lectures covering the ArrayList versus LinkedList implementations of the Java List interface, I spent an entire lecture on JUnit and live-coded a performance test suite that produced actual data to back up our discussions of big-O complexity. The best part of all? They learned about JIT compilation in the JVM firsthand because it completely blew apart the expected test results.

varjag 1y ago

Sudokugate flashbacks.

zeroCalories 1y ago

The reason you study ds&a is because it's a difficult skill. Testing is incredibly straightforward and easy to learn. The reason tests suck is because the infra suck, the eng is lazy, or both.

JackSlateur 1y ago

I think the author is mislead Let's grab a simple use case: some basic CRUD http API. Easy, you say, no need to know fancy stuff ! Just test it and that's all. You do your test, all good, you can roll in production ! But sadly, in production, you have multiple users (what an idea ..). Suddenly, your CRUD api has become a concurrent system. Suddenly, you have data corruption, because you never thought about anything about that, and "your tests were green". Algorithms are the backbone tools of programming. Knowing them help us, ignoring them burdens us.

wubrr 1y ago
I mean, in your example you just have an incomplete test suite. (Though writing a complete one is often unrealistic) While understanding algorithms and data structures is important, the only way you really know how well it works, and how well it's implemented is by thoroughly testing it. There are an infinite amount of clever algorithms out there with terrible implementations. You need both.
- JackSlateur 1y ago
  Testing concurrency is extremely hard For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Or is it because one query ran 10ms earlier and, thus, you avoided the issue ? I truly wonder if there is real world tests around this; I bet there is only algorithm and fuzzing;
  - wubrr 1y ago
    > Testing concurrency is extremely hard Writing a non-trivial concurrent system based on your understanding of the 'algorithm' , without relying on testing is much harder. > I truly wonder if there is real world tests around this Of course there are. There are many tools, methods, and test suites out there for concurrency testing, for almost any major language out there. Of course, understanding your algorithm, and the systems involved is required to write a proper test suite. > For instance, get sql queries; You ran them, and you have no issue; Is your code sane ? Take those queries and run them 1000x+ times concurrently in a loop. That will catch most common issues. If you want to go a step further you can build a setup to execute your queries in any desired order.

jsd1982 1y ago

Are we assuming that "testing" is limited to only exercising the single-threaded behavior of a function? I'm curious how others approach effective testing of multi-threaded behavior.

IvyMike 1y ago

Always gonna have to side with Peter Norvig on this one: https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-work.html https://pindancing.blogspot.com/2009/09/sudoku-in-coders-at-... > They said, “Look at the contrast—here’s Norvig’s Sudoku thing and then there’s this other guy, whose name I’ve forgotten, one of these test-driven design gurus. He starts off and he says, “Well, I’m going to do Sudoku and I’m going to have this class and first thing I’m going to do is write a bunch of tests.” But then he never got anywhere. He had five different blog posts and in each one he wrote a little bit more and wrote lots of tests but he never got anything working because he didn’t know how to solve the problem. I actually knew—from AI—that, well, there’s this field of constraint propagation—I know how that works. There’s this field of recursive search—I know how that works. And I could see, right from the start, you put these two together, and you could solve this Sudoku thing. He didn’t know that so he was sort of blundering in the dark even though all his code “worked” because he had all these test cases.

IvyMike 1y ago
More context, from an earlier HN comment: https://news.ycombinator.com/item?id=3033446 https://news.ycombinator.com/item?id=3033446
pncnmnp 1y ago
I love what Norvig said. I can relate to it. As far as data structures are concerned, I think it's worth playing smart with your tests - focus on the "invariants" and ensure their integrity. A classic example of invariant I can think of is the min-heap - node N is less than or equal to the value of its children - the heap property. Five years from now, you might forget the operations and the nuanced design principles, but the invariants might stay well in your memory.
roxolotl 1y ago
This totally misses the point of the article. The article agrees that knowing when a problem is a data structure and algo problem is a key strength. The article also isn’t saying that all development should be done TDD. The point of the article is that knowing how to test well is more useful than memorizing solutions to algo problems. You can always look those up.

29athrowaway 1y ago

Learning both is not mutually exclusive.

sakesun 1y ago

Just realise that I have been reading his blog for two decades already.

rossant 1y ago
Same. Love his blog.

JoeAltmaier 1y ago

We wrote a conferencing app and server (years before Zoom). Tested the server by having automated headless apps run in gangs, a hundred at a time, hopping from conversation to conversation, turning mic and camera on and off, logging out and logging back in. Used it for years, the Bot Army we called it. Responsible for our rock-solid quality reputation. Not API design or test classes or constraints or anything. Just, trying the damn thing, in large cases, for a long time. When it ran an hour, we celebrated. When it ran overnight, we celebrated. When it ran a week we celebrated, and called that good enough.

rossant 1y ago
How much work was it to go from 1 hour to 1 week? How many issues have you discovered, what were they? Genuinely interested.
- JoeAltmaier 1y ago
  It took a big fraction of our energy to get to each new stage. Always it was something new and unexpected. It was quite a while ago, but lets see what I remember. Some good fraction were mismatches between the app (bot) state and the server state. A bot would be expecting a message and stall. The server thought it had said enough. The app side used a lot of libraries, which it turns out are never as robust as advertised. They leak, race, are very particular about call order. Have no sense of humor if they're still connecting and a disconnect call is made, for instance. The open source server components were fragile. In one instance, the database consistency library had an update where, for performance, a success message was returned before the operation upstream was complete. Which broke, utterly, the consistency promise that was the entire point of using that product. A popular message library created a timer on each instantiation. Cancelled it, but in typical Java fashion didn't unlink it. So, leak. Tiny, but you do it enough times, even the biggest server instance runs out of memory. We ran bots on Windows, Linux, even a Mac. Their network libraries had wildly different socket support. We'd run out of sockets! They got garbage collected after a time, but the timer could be enormous (minutes). Our server used a message-distribution component to 'shard' messages. It had a hard limit on message dispatching per second. I had to aggregate the client app messages (we used UDP and a proprietary signaling protocol) to drop the message rate (ethernet packet rate) by an order of magnitude. Added a millisecond of latency, which was actually important and another problem. Add the usual Java null pointers, order-dependent service termination rules (never documented), object lifetime surprises. It went on and on. Each doubling of survival-time the issues got more arcane and more interesting. Sometimes took a new tool or technique to ferret out the problem. To be honest, I was in hog heaven. Kept my brain plastic for a long time.

OfflineSergio 1y ago

I think they enough unnecessary content in universities that there be room for both DSA and testing.

rr808 1y ago

Do the people who love testing run JS or Python or something that compiles really quick? I've worked on some big projects and just to make a change, compile and run a unit test often takes 5-15 minutes. TDD works great if you have a trivial library in an interpreted language.

yakshaving_jgt 1y ago
Understanding how to write tests in an economical fashion is a skill in itself. With care, you can write fast (and effective) tests in Haskell (and, I’m sure, every language ever).
markmark 1y ago
Testing doesn't imply TDD.

ngcc_hk 1y ago

Can a program function without data, algorithm or testing or users or programmers. Now which one is better … sorry what rubbish question is that. All are important. And limited time you still have to consider all these … noth8ng is better.

quotemstr 1y ago

> I see new learners asking about “DSA” a lot. I've noticed this "DSA" acronym appearing overnight. I can't recall people using it this much (at all actually) even six months ago. Where did it come from? Why do we suddenly need a term to talk about the concept?

burch45 1y ago
That is the standard acronym for the course in American universities and has been for many decades.
- quotemstr 1y ago
  Maybe so, but that doesn't explain why have people suddenly started using it more.

curtisszmania 1y ago

[dead]

lolive 1y ago

Data structures are overrated. One day, a trainee told me that maybe a Set would be better than an array in one of my codes. I fired this ignorant immediately. #noOverengineering #arrayIsGoodEnough

teo_zero 1y ago

As usual, the title doesn't reflect the message of the article. While critical against learning tons of algorithms and structures by heart, the author advocates understanding what they do and why you would want to use them. This is what it means to master TSA. The last part denounces that testing is not tought enough, and learning it may be beneficial for your career. It's never said that testing is better than TSA, just that we would need more.

Arun2009 1y ago

I have always thought of DSA as a proxy for a subset of general software development skills: the ability to translate a problem into computer science or programming terms, implement it in code, and argue that the implementation is correct and efficient. Skill in solving DSA problems can signal both an aptitude for absorbing computer science knowledge in general and a capacity for solving problems through programming. It's not the whole thing, but it's certainly an important component. It’s not unlike a research mathematician being expected to solve quadratic equations. He may not need them in his day-to-day work, but with a little preparation he should be able to handle them. If he struggles with quadratic equations in an interview where such knowledge is expected, that would raise a red flag about his training.

solumos 1y ago
> It’s not unlike a research mathematician being expected to solve quadratic equations. He may not need them in his day-to-day work, but with a little preparation he should be able to handle them. If he struggles with quadratic equations in an interview where such knowledge is expected, that would raise a red flag about his training. The absurdity of this indicates a big part of what’s wrong with modern SWE interviews. You wouldn’t ask a research mathematician to derive the quadratic formula, and you wouldn’t give a writer a spelling test — passing or not isn’t related to their aptitude or proficiency. Additionally, imagine interviewing a complex analyst and quizzing them only on calc 1 — sure, it may be a course that they were expected to take at some point, but it really has little to do with what their actual work entails, and so half of the interview they’d be trying to slot what the proper layer of abstraction is in the limited context of the problem.
pezgrande 1y ago
Maybe in some companies. At "Enterprise" level your control-alt-f skills are more valuable (for your average IC) based on my experience.

jillesvangurp 1y ago

Well, you need both. You are going to suck at implementing algorithms if you don't learn how to test they actually work as intended. Most algorithms are used in library form. Unless you are writing those libraries, you probably should not be reinventing a lot of wheels. So there's a valid argument there that most of the stuff you learn as part of your computer science courses, you will not be implementing over and over again (if ever). So, you could argue that testing is a more universal skill that you need either way. But algorithms can come up once in a while. And it helps if you can guesstimate complexity of various algorithms and make some trade offs over picking one or the other. The skill you learn in college is not any particular algorithm but a broad knowledge of which mainstream ones are there, how they work, their tradeoffs, and the skill of implementing those or similar algorithms. You gain the skill of good judgment, being able to figure out how stuff works, and general intuition of how things are done at a high level. I've never developed a file system. But I know tree data structures such as b trees and red black trees have something to do with it. It's been decades since I looked at that stuff. But I could read up in an afternoon or so if it comes up. That doesn't qualify to start working on a file system. But, I don't think that's going to come up anyway. I have plenty of other things to do. I do dabble a bit with search algorithms once in a while. More of a hobby than work related. But there's some room in that space for being able to do some basic things with algorithms instead of using some prefab search product.

mekoka 1y ago

Many programmers think that the way DS&A work is that one beautiful spring morning, some requirements fall onto your lap where the words "nodes" and "edges" are neatly circled, thus marking a key moment in your life where you can finally put to use that old dusty algorithm book. Meanwhile, trees and graphs have been flying up to your face, slapping you left and right, every single day of your career and you kept repeating to yourself "see, you don't need all that DS&A stuff". I think the article is unnecessarily trying to contrast two orthogonal and separately useful software development concerns. DS&A are about having the most efficient tools for the specific job. Testing is about ensuring that results match expectations. But I'd be surprised by anyone who only learns DS&A in theory and also naturally develops the related, but subtler skill of recognizing classes of problems that match up to particular algorithms. It's an almost tacit skill, which many programmers don't have despite knowing some of the classic DS&A, because it requires to linger a bit longer in the material. And that's the real gap leetcode type challenges try to bridge. The more you do them, the more you reinforce an intuitive understanding of involved structures and techniques, including some of the subtler properties and oft undocumented corner cases. What looks like "memorization" to some is actually more of an almost indelible grokking, which you carry with you in professional programming. Admittedly, people who only do leetcode type challenges and never write software in the real world don't know which parts of these challenges are useless (in my opinion, the puzzles and lateral thinking parts). But people who never do these challenges also know squat about how useful they can be at honing one's "real life" problem solving and coding skills.

specialist 1y ago
> ...unnecessarily trying to contrast two orthogonal and separately useful software development concerns. > ...only do leetcode type challenges and never write software in the real world... Respectfully, I agree with OC's point about testing vs implementation. (Both are programming.) It's been a (long) while since I've had peers who regularly, usefully tested their own work. Like this OC, I too have struggled to articulate why leetcode has tiny IRL relevance. How's this: A useful distinction may be implementation vs usage. Of course, a tiny handful of people are entrusted with implementing libraries of algorithms, so should be properly vetted for that work. Ditto crypto, parsing, and other misc arcane arts. OC's point remains that leetcode hazing ignores the majority of actual work as a programmer. Such as fixing bugs, testing, modeling, documenting, enduring meetings, etc. -- IIRC, I haven't implemented a sort algorithm, either in anger or for fun, since my BASIC & Pascal days. Why would I? But I have recast real world problems as a Traveling Sales Person problems a handful of times. As you well know, modeling using well known data structures, to enable using stock algorithms, is a big part of the job. I'd rather interviews verified applicable skills. Such as data modeling, sequence diagrams, etc. And maybe some lightweight arch like (from the hip) caching, serialization, indexing, queuing and backpressure, locks, and validation (and whatever other bog standard stuff is immediately relevant). -- (Oops: I did recently implement topo sort, just to understand it better, even though I was using a ready-baked solution. Just like +25 years ago I implemented (poorly) taboo, simulated annealing, etc. during my brief optimization kick.)

lunias 1y ago

"If my Grandmother had wheels, she would have been a bike."

gdubs 1y ago

I find that test-driven advocates border on a religious way of thinking about software development, and I also find it sucks joy out of coding. Does that mean I don't think tests are valuable – no, that's not what I'm saying. There are critical pieces of code where testing will bring you more joy because you'll catch very bad bugs that would have really ruined your day. But as a way of constructing an application, it just has always felt really dull and uninspiring. I know people will vehemently disagree with this. But, sometimes you gotta write the novel once to know what story you're telling. Then you can go back, edit, put tests in the critical places.

BrenBarn 1y ago

This is a dangerous statement, although I don't think that was the author's intent: > Before I get to other opinions, let me make clear that anything you can do to help you get a job is a good thing to do. There are lots and lots of very evil things that will help you get a job, especially with companies that want to do evil things. Those are definitely not good things to do. I think the author intended a more restricted context like "any skill you can learn that will help you get a job is a good skill to learn from a technical perspective" (i.e., it might still be ethically wrong). This may seem pedantic but I've known some young graduates recently who went to work for very evil companies, and I think it's easy for impressionable minds to read statements like this and think "Yeah, see, everyone does it, who cares" without filling in the ethical background that the author assumes.