4 ms·
Very much not the parent, but as a heavy R user, I don't think either of them quite nail the way dplyr and the tidyverse work. The thing about dplyr is... it's
by cwyers 6y ago
Very much not the parent, but as a heavy R user, I don't think either of them quite nail the way dplyr and the tidyverse work. The thing about dplyr is... it's just functions. Okay, so, it's functions that leverage features R has (notably lazy evaluation and non-standard evaluation). But it's just functions. All you need is a function that takes a data frame and returns a data frame. So you can take a function out of the R standard library, you can take a function from a package written before dplyr came around, you can take a function from a recent non-tidyverse package, you can write your own function... it's all just functions.
In DataFramesMeta.jl, though, you have a macro, and everything runs inside that macro. So if you want to take something that isn't a part of DataFramesMeta.jl... here's an example. Let's say you want to take the popular mtcars dataset, and get the five cars with the best gas milage. In dplyr, that goes
mtcars %>% arrange(mpg) %>% head(5)
arrange is a function from the dplyr package, head is a function from the standard library, but they both work seamlessly together.
DataFramesMeta.jl lets you work in a pipe-forward fashion, but (at last I knew, at least, it's been a while since I played with it), you couldn't use the Julia head function within a DataFramesMeta.jl pipeline. You have to do your data transformations, assign to a variable, and then get the head of that variable.
Which, okay, probably doesn't sound like a big deal. But I think it gets at the heart of what efforts to do something Tidyverse-like in other languages (Python and Julia, mostly) really miss. The key value proposition of the Tidyverse in R is that it is very composable and very extensible. That means, if you are trying to solve something in a Tidyverse way, you can probably find something that works for you. If you are doing financial analysis? Get tidyquant. If you're doing time series analysis, the tidyverts packages are for you. And it all works because there is so little friction involved in writing your own functions that extend the functionality of Tidyverse packages. Yes, dplyr is a useful querying DSL in its own right, but you can find a bunch of SQLish query languages, and they're all some degree of fine. Query.jl or DataFramesMeta.jl might expose a useful querying DSL for data frames, but they don't seem to me to be built to support building a whole ecosystem like dplyr and the Tidyverse are.
- phillc73 6y agoThat's a really good point that I'd not really thought about. I'd never really considered the difference between calling just functions versus macros. Thinking about Query.jl and DataFramesMeta.jl, and I am for sure not an expert in either, I can't specifically speak to your `head` example, but other base functions can be combined with macros. For example, see the LINQ examples from DataFramesMeta.jl[1] where `mean` is being used. Or again the LINQ style examples in Query.jl[2], where `descending` is used in the first example, or `length` later in the Grouping examples. Is that the kind of thing you meant? For whatever reason, with the way my brain is wired, the LINQ style of query just works for me. I have never directly used LINQ, but do have some SQL experience. In fact, I wrote some dinky little wrapper functions[3] around duckdb[4] so I could directly query R dataframes and datatables with SQL using that backend, rather than sqldf[5]. [1] https://juliadata.github.io/DataFramesMeta.jl/stable/#@linq-and-other-chaining-macros https://juliadata.github.io/DataFramesMeta.jl/stable/#@linq-... [2] https://www.queryverse.org/Query.jl/stable/linqquerycommands/ https://www.queryverse.org/Query.jl/stable/linqquerycommands... [3] https://github.com/phillc73/duckdf https://github.com/phillc73/duckdf [4] https://duckdb.org/ https://duckdb.org/ [5] https://cran.r-project.org/web/packages/sqldf/index.html https://cran.r-project.org/web/packages/sqldf/index.html