4 ms·
To clarify: By "static analysis" I mean anything you can learn from studying source code without running it. (Though, with Agda this isn't a clear distinction;
by skybrian 8y ago
To clarify:
By "static analysis" I mean anything you can learn from studying source code without running it. (Though, with Agda this isn't a clear distinction; maybe we could talk about what we can extrapolate about whatever the compiler didn't evaluate.)
I was speaking of testing broadly to include manual QA and running benchmarks. A continuous benchmark will give a graph of the system's performance for certain actions. It's typically not pass/fail, but if the developers notice a regression they can investigate. These slowdowns just aren't correctness bugs as commonly understood and many places don't notice performance issues until after the code goes live and it's noticed via monitoring. (Benchmarks are tricky to write, tricky to run consistently, don't generalize, and miss a lot of things, but nonetheless measure things that static analysis doesn't even try to do.)
"Vast majority": I guess this depends on what you put in the denominator? I can certainly believe that Agda programmers don't see performance issues. But I'll confess ignorance: is this used for anything other than doing math?
I'm thinking more along the lines of: if a more mainstream language had totality checking, what bugs would this actually find in the code that runs in data centers? Plenty, I'm sure. But I can think of lots of things it wouldn't. It's one method of bug-finding among many.
- lmm 8y ago> I was speaking of testing broadly to include manual QA and running benchmarks. A continuous benchmark will give a graph of the system's performance for certain actions. It's typically not pass/fail, but if the developers notice a regression they can investigate. These slowdowns just aren't correctness bugs as commonly understood and many places don't notice performance issues until after the code goes live and it's noticed via monitoring. (Benchmarks are tricky to write, tricky to run consistently, don't generalize, and miss a lot of things, but nonetheless measure things that static analysis doesn't even try to do.) Well we've got to compare similar levels of effort. I think most "normal" teams would find it easier to adopt a more detailed type system than to set up a continuous benchmark and use it effectively. I could be wrong. > "Vast majority": I guess this depends on what you put in the denominator? I can certainly believe that Agda programmers don't see performance issues. But I'll confess ignorance: is this used for anything other than doing math? > I'm thinking more along the lines of: if a more mainstream language had totality checking, what bugs would this actually find in the code that runs in data centers? Plenty, I'm sure. But I can think of lots of things it wouldn't. It's one method of bug-finding among many. I don't know so much about Agda specifically. I agree with thinking about mainstream code that runs in data centers. My thought process was: of the code-runs-too-long bugs I've seen in my career, how many of those could have happened if I'd been working in Idris with totality checking on? I can only think of one (which was due to regex catastrophic backtracking), and I can think of several that would have been caught (the case where the test used the wrong overload of the function being the most production-impactey one). My experience is that while other methods of finding bugs do exist, the cost/benefit for type systems is just so much better than anything else that I wouldn't even think about using any other technique until I'd tried type systems first.