3 ms·
This is extremely interesting. This paper does a few different things that are worth separating: 1. Structures knowledge about common statistical tests, the as
by exp1orer 8y ago
This is extremely interesting. This paper does a few different things that are worth separating:
1. Structures knowledge about common statistical tests, the assumptions they need to satisfy, and the kinds of problems for which they are appropriate.
2. Models this as a constraint satisfaction problem.
3. Creates a DSL to write specifications for statistical testing.
4. Develops an output format for statistical testing based on (3).
5. Provides a python package that implements the above.
IMO (1) is by far the most fundamental of these, but the paper spends most of its time describing (3). I'll admit to not fully understanding (2), what the alternatives were, and why it's an appropriate choice - I wish there were more on that in the paper as well.
If the approach to (1) were described in more depth and published in a usable way outside of the implementation in (5), I could see it being broadly useful and leading to an ecosystem of different (competing) implementations. As it stands, the work on (1) is spread across a few functions in solver.py.
Looking more broadly, statistics-as-an-HCI-problem is fascinating, and I think this is a promising start. I'd also love to see more attention paid to (4), the output format. Most "doing harm with statistics", I believe, comes from misunderstanding/misapplying results, so looking at the full workflow / user story is critical. The focus of this paper is instead making the input easier, which is also obviously important.
emjun, so glad to see you in the comments here! What's the best way for interested folks to contribute to this project?
- emjun 7y agoHi! I, too, think statistics-as-an-HCI problem is a cool framing that allows for new methods/solutions :) Your observations and distinctions about what Tea does are great. I am currently trying to fix some bugs/make improvements from the deadline push. After that, I plan to restructure the internals to make the logic/reasoning/implementation easier to follow and extend. I would love contributors! It would be great if you could watch, open an issue, or open a pull request on Github (https://github.com/emjun/tea-lang https://github.com/emjun/tea-lang). If there are enough interested collaborators, it might be worth opening up a gitter or slack group? In the meantime, if you'd like to chat more, please feel free to email me! (<my username> at cs.washington.edu)