3 ms·
My favorite example of this is Hadley Wickham's work on tidy data [1, 2]. Before Hadley, tools for streamlining data manipulation workflows had largely fallen
by civilized 3y ago
My favorite example of this is Hadley Wickham's work on tidy data [1, 2].
Before Hadley, tools for streamlining data manipulation workflows had largely fallen stagnant. Even the best languages and tools were a chore to use. People's data prep processes tended to be idiosyncratic and hard to understand, even though there was nothing essentially new or deep about what they were doing. Data prep was an ugly chore best left to interns and grad students. It was kept out of sight so the big wigs could focus their slide decks on more interesting things.
Traditional academic statisticians and computer scientists had a blind spot for this issue. They didn't consider it a hard problem, which is what academics are supposed to work on, and discouraged Hadley from working on it early in his academic career. But Hadley persisted because he knew it was an important problem. Streamlining time-consuming chores liberates an enormous amount of human capital for the most meaningful work. (Which in some cases can be improving the data still further, opening up new vistas of insight the big wigs otherwise never would have even been aware of!)
Hadley's approach has not only revolutionized R programming, but has also quietly influenced the whole modern analytics ecosystem. It's become too good to ignore. Even for those who do most of their work in another language, with tools like pandas that don't follow the tidy philosophy, some end up trying to structure their workflows like Hadley's tools do, because it's so intuitive and compelling [3].
[1] http://vita.had.co.nz/papers/tidy-data.pdf http://vita.had.co.nz/papers/tidy-data.pdf
[2] https://imstat.org/2014/09/04/data-science-how-is-it-different-to-statistics%E2%80%89/ https://imstat.org/2014/09/04/data-science-how-is-it-differe...
[3] https://stmorse.github.io/journal/tidyverse-style-pandas.html https://stmorse.github.io/journal/tidyverse-style-pandas.htm...