3 ms·

Veritasium has a great video[1] about people’s cognitive bias towards using examples that prove their mental model, rather than using examples that disprove the

Veritasium has a great video[1] about people’s cognitive bias towards using examples that prove their mental model, rather than using examples that disprove their mental model. But if you want to actually confirm you have accurate understanding of something, testing examples that don't fit your metal model is the workable methodology (i.e. use a null hypothesis). Testing examples that do match your mental model only proves your model partially matches the actual model, but it does very little to actually identify misunderstandings, or improve your understanding. [1] https://youtu.be/vKA4w2O61Xo https://youtu.be/vKA4w2O61Xo

jerf 3y ago
It's a common programming error too. I often see automated test cases that only test the happy path, but you ought to test that errors error the way you expect and desire too. These tests fail the first time I run them as often as the happy path cases do.
- Spivak 3y ago
  This to me gets to the fundamental nature of testing. If errors are part of your test suite then they're happy path too. It's just as much a part of your API surface as any other return value. But if you don't have (or want) callers that depend on your specific behavior in error situations then I wouldn't bother testing them. It's my contract that I can and will change the behavior arbitrarily and you can't be mad at me for it.
Stratoscope 3y ago
So many great videos on Veritasium, and that is one of my favorites. Anyone who is not familiar with the channel should check it out. Of course, there is this counterpoint... https://www.youtube.com/watch?v=cY_o4A1wzsg https://www.youtube.com/watch?v=cY_o4A1wzsg
op00to 3y ago
I was taught to use the null hypothesis in a US high school science lesson on the scientific method in the 90s. It never clicked to me “why” until your explanation. Funny how that stuff works.
cxr 3y ago
Previously: "A Quick Puzzle to Test Your Problem Solving". 2015 July 2. 288 comments. <https://www.nytimes.com/interactive/2015/07/03/upshot/a-quick-puzzle-to-test-your-problem-solving.html https://www.nytimes.com/interactive/2015/07/03/upshot/a-quic...> You'll need to paste this snippet into the JS console in your browser (or make a bookmarklet out of it and click it in order to reveal the article) because something on nytimes.com has caused it to break at some point in the last 8 years: document.querySelector(".g-result.g-noText").style.display = "block" HN comments: <https://news.ycombinator.com/item?id=9818310 https://news.ycombinator.com/item?id=9818310>

jerf 3y ago

It's a common programming error too. I often see automated test cases that only test the happy path, but you ought to test that errors error the way you expect and desire too. These tests fail the first time I run them as often as the happy path cases do.

Spivak 3y ago
This to me gets to the fundamental nature of testing. If errors are part of your test suite then they're happy path too. It's just as much a part of your API surface as any other return value. But if you don't have (or want) callers that depend on your specific behavior in error situations then I wouldn't bother testing them. It's my contract that I can and will change the behavior arbitrarily and you can't be mad at me for it.

Stratoscope 3y ago

So many great videos on Veritasium, and that is one of my favorites. Anyone who is not familiar with the channel should check it out. Of course, there is this counterpoint... https://www.youtube.com/watch?v=cY_o4A1wzsg https://www.youtube.com/watch?v=cY_o4A1wzsg

op00to 3y ago

I was taught to use the null hypothesis in a US high school science lesson on the scientific method in the 90s. It never clicked to me “why” until your explanation. Funny how that stuff works.

cxr 3y ago

Previously: "A Quick Puzzle to Test Your Problem Solving". 2015 July 2. 288 comments. <https://www.nytimes.com/interactive/2015/07/03/upshot/a-quick-puzzle-to-test-your-problem-solving.html https://www.nytimes.com/interactive/2015/07/03/upshot/a-quic...> You'll need to paste this snippet into the JS console in your browser (or make a bookmarklet out of it and click it in order to reveal the article) because something on nytimes.com has caused it to break at some point in the last 8 years: document.querySelector(".g-result.g-noText").style.display = "block" HN comments: <https://news.ycombinator.com/item?id=9818310 https://news.ycombinator.com/item?id=9818310>