Wow. You ever get your mind blown three different ways at once? I don't know if I'm more impressed with the depth of ideas in this paper, the exquisite clarity

Wow. You ever get your mind blown three different ways at once? I don't know if I'm more impressed with the depth of ideas in this paper, the exquisite clarity of its presentation, or the glimpse this journal gives into the science of the future: open, participatory, technologically empowered, and anchored in the enrichment of human understanding. This is great and important work. I was looking at the previous paper on feature visualisation (https://distill.pub/2017/feature-visualization/ https://distill.pub/2017/feature-visualization/) and I couldn't help but notice the parallels between feature examples in neural networks and test cases in traditional programming. Examples drawn from real datasets appear in both, as well as generating examples using optimisation processes (search-based software testing[0]), and optimising from real data (test data augmentation[1]). I even found something similar to the diversity-maximisation approach[2]. There are also some related ideas in the functional programming world that combine optimisation with constraints on the input domain (targeted property-based testing[3]) and do a similar kind of human-scale input reduction (counterexample reduction[4]). More generally, maybe it makes sense to think of testing and interpretation as complementary ideas. Testing says "I think I understand this function's behaviour, but I want to examine its output and compare it against what I expect." Interpretation says "I think I understand this function's output, but I want to examine its behaviour and compare it against what I expect." Test cases are inputs that generate interesting outputs, and feature examples are inputs that generate interesting behaviour. In either case, the goal is to minimise the inputs and maximise interestingness, which it seems is best understood in terms of the relationship between behaviour and output rather than either one alone. I'm curious if anyone's looked into this. Maybe there are some neat ways to apply techniques from one to the other, or even combine the two? [0] https://pdfs.semanticscholar.org/67a9/ca5a33e3ab4c2300cdcfaafdfa6aeb989eb0.pdf https://pdfs.semanticscholar.org/67a9/ca5a33e3ab4c2300cdcfaa... [1] http://crest.cs.ucl.ac.uk/fileadmin/crest/sebasepaper/YooH08.pdf http://crest.cs.ucl.ac.uk/fileadmin/crest/sebasepaper/YooH08... [2] https://pdfs.semanticscholar.org/a261/87634f842919ef53d0da4fc3074f7ecb5cde.pdf https://pdfs.semanticscholar.org/a261/87634f842919ef53d0da4f... [3] http://proper.softlab.ntua.gr/papers/issta2017.pdf http://proper.softlab.ntua.gr/papers/issta2017.pdf [4] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf