4 ms·

The Tidyverse is a true gem in modern programming that deals with data. This release feels like a nice incremental improvement rather than a revolution. Adding

The Tidyverse is a true gem in modern programming that deals with data. This release feels like a nice incremental improvement rather than a revolution. Adding lubridate as a core package will save me a few keystrokes. I also did not know about the conflicts package, but will certainly be using it in the future. I'm a bit surprised that other "better" programming languages haven't done more to replicate the Tidyverse. The Tidyverse is a masterclass in metaprogramming, and has a pretty big following. While R has some warts as a language, I'm not sure I've seen anything comparable to Tidyverse in terms of scope and influence. I see Julia trying with the Dataframes ecosystem, but it's got a ways to go before that feels cohesive like Tidyverse. If I could wave a magic wand, I'd get Hadley and the RStudio crew really excited about Julia or Racket. R does have warts, and I dream of the day when something better is as good.

AuthorizedCust 4y ago
> This release feels like a nice incremental improvement rather than a revolution. Look at the big picture, not simply the delta from the prior .x.y release. The Tidyverse feels like a product that evolves over time, rather than a traditional approach that jumps versions. Major changes sometimes come in seemingly minor releases. I feel that 2.0 simply means the evolution has gone on enough to put a retrospective marker in the sand. For something geared more to data science and exploration, rather than production use, I’m good with the evolutionary approach.
- Breza 3y ago
  Well said, I completely agree with you!
- peatmoss 4y ago
  I agree 100% that this is a good thing. I had intended this as a compliment that Tidyverse had achieved this level of maturity, but failed in my under-caffeinated state. Had Tidyverse 2.0.0. been a revolution, I'd have been worried.
leephillips 4y ago
“Tidier.jl is a 100% Julia implementation of the R tidyverse mini-language in Julia.” https://github.com/TidierOrg/Tidier.jl https://github.com/TidierOrg/Tidier.jl
- peatmoss 4y ago
  This illustrates the point perfectly. Julia is attempting this and has a beachhead with Dataframes.jl. Confusingly though, Tidier.jl isn't really analogous to R's Tidyverse. It's more like one of a handful of meta-packages around Dataframes.jl. Then there are Grammar of Graphics (ggplot was Tidyverse's first star) style plotting libraries that Julia has been building. I'm probably most excited about Algebra of Graphics (https://github.com/MakieOrg/AlgebraOfGraphics.jl/ https://github.com/MakieOrg/AlgebraOfGraphics.jl/) as part of the Makie Plots ecosystem. It does still feel a bit like Julia community can't decide between following Matplotlib or R's Grid/Ggplot approach. The seeds of a Tidyverse for Julia are there, but it'll take some time to achieve the consistency and maturity of the original Tidyverse.
  - martinsmit 4y ago
    I think DF.jl works remarkably well as a "fits in RAM" dataframe backend, but I think it just lacks in usability and integratedness with the wider Julia ecosystem. Or rather, the wider ecosystem isn't as mature in key data analysis areas. In particular, as you mention, plotting is one of the evolving parts of the ecosystem. Plots.jl is fine, Makie is powerful but very DIY, and AoG is slick but unwieldy. ggplot2 is far from perfect, but it works so well due to its maturity and integration with the rest of the Tidyverse. In my ideal world, there would be a DataFrames.jl wrapper to provide nice (not just nicer like the two DFM.jl packages) syntax, and a powerful high-level plotting package (Makie is powerful but syntax is low level, Plots is mid on both) which is heavily integrated with the wrapper package. Admittedly, I'm not a data scientist (anymore) so I don't follow the new developments in the dataviz scene much. If something like this exists then I would love to find it. I wonder what my ideal syntax would look like anyway. Maybe something close to Tidyverse but with symbols as column names `:col_name` for ambiguity reasons.
0xDEF 4y ago
>I'm a bit surprised that other "better" programming languages haven't done more to replicate the Tidyverse Python has a very weak metaprogramming story so it is hard if not impossible to replicate something like Tidyverse in that. If Ruby had succeeded in data science instead of Python it would almost certainly be with Tidyverse-like abstractions.
_Wintermute 4y ago
I don't know if it's impressive or worrying regarding how much Posit/RStudio as a company have completely taken over a programming language. I have colleagues who write R code all day, and they don't know the difference between RStudio the IDE and R the language. They can't perform basic tasks such as defining a function, or adding a column without the tidyverse.
- fbdab103 4y ago
  I see that same thing all the time. RStudio = R, and they have no concept of the language without the IDE. Which is fine, I guess? They are not professional programmers, but people with a job to do. It is just frustrating to me when they RShiny all the things, and I have to pickup the pieces because the spaghetti has collapsed under its own weight.
  - peatmoss 4y ago
    I see it a little bit more like a meritocracy in that space that RStudio has largely won. Microsoft has certainly made an attempt to become the face of R. Remember when they purchased Revolution Analytics and started baking it into everything? Or how about Oracle making a play on Graalvm with "FastR"? RStudio has somewhat indefatigably looked for pain points that R users had back in the olden days. And yes, I remember R back when lattice plots were the hottest game in town. Not only has the Tidyverse made R infinitely more pleasant to work with, the community around Tidyverse has made the R community infinitely less insufferable for newcomers. The obtuse responses to beginners on the old R mailing lists were legendary. My preferred R IDE is emacs, and I'm not upset with what RStudio has done. When I interact with devtools, I don't feel that I'm wanting for functionality. This is in spite of the fact that RStudio has heavily integrated devtools functionality into their IDE, and could arguably have made devtools less easy to work with using generic tooling. I understand the mediocrity of many RShiny applications in terms of how they are written. But, another way of looking at all those dashboards and interactive applications, is that they are AMAZING prototypes written by scientists, grad students, journalists, etc., who would have never been able to build them some other way. If we forced everyone to write these apps in Rust, of course the average quality would be higher, but we'd also have way fewer of them. Democratization of technology means that its users will be more... medium.

AuthorizedCust 4y ago

> This release feels like a nice incremental improvement rather than a revolution. Look at the big picture, not simply the delta from the prior .x.y release. The Tidyverse feels like a product that evolves over time, rather than a traditional approach that jumps versions. Major changes sometimes come in seemingly minor releases. I feel that 2.0 simply means the evolution has gone on enough to put a retrospective marker in the sand. For something geared more to data science and exploration, rather than production use, I’m good with the evolutionary approach.

Breza 3y ago
Well said, I completely agree with you!
peatmoss 4y ago
I agree 100% that this is a good thing. I had intended this as a compliment that Tidyverse had achieved this level of maturity, but failed in my under-caffeinated state. Had Tidyverse 2.0.0. been a revolution, I'd have been worried.

leephillips 4y ago

“Tidier.jl is a 100% Julia implementation of the R tidyverse mini-language in Julia.” https://github.com/TidierOrg/Tidier.jl https://github.com/TidierOrg/Tidier.jl

peatmoss 4y ago
This illustrates the point perfectly. Julia is attempting this and has a beachhead with Dataframes.jl. Confusingly though, Tidier.jl isn't really analogous to R's Tidyverse. It's more like one of a handful of meta-packages around Dataframes.jl. Then there are Grammar of Graphics (ggplot was Tidyverse's first star) style plotting libraries that Julia has been building. I'm probably most excited about Algebra of Graphics (https://github.com/MakieOrg/AlgebraOfGraphics.jl/ https://github.com/MakieOrg/AlgebraOfGraphics.jl/) as part of the Makie Plots ecosystem. It does still feel a bit like Julia community can't decide between following Matplotlib or R's Grid/Ggplot approach. The seeds of a Tidyverse for Julia are there, but it'll take some time to achieve the consistency and maturity of the original Tidyverse.
- martinsmit 4y ago
  I think DF.jl works remarkably well as a "fits in RAM" dataframe backend, but I think it just lacks in usability and integratedness with the wider Julia ecosystem. Or rather, the wider ecosystem isn't as mature in key data analysis areas. In particular, as you mention, plotting is one of the evolving parts of the ecosystem. Plots.jl is fine, Makie is powerful but very DIY, and AoG is slick but unwieldy. ggplot2 is far from perfect, but it works so well due to its maturity and integration with the rest of the Tidyverse. In my ideal world, there would be a DataFrames.jl wrapper to provide nice (not just nicer like the two DFM.jl packages) syntax, and a powerful high-level plotting package (Makie is powerful but syntax is low level, Plots is mid on both) which is heavily integrated with the wrapper package. Admittedly, I'm not a data scientist (anymore) so I don't follow the new developments in the dataviz scene much. If something like this exists then I would love to find it. I wonder what my ideal syntax would look like anyway. Maybe something close to Tidyverse but with symbols as column names `:col_name` for ambiguity reasons.

0xDEF 4y ago

>I'm a bit surprised that other "better" programming languages haven't done more to replicate the Tidyverse Python has a very weak metaprogramming story so it is hard if not impossible to replicate something like Tidyverse in that. If Ruby had succeeded in data science instead of Python it would almost certainly be with Tidyverse-like abstractions.

_Wintermute 4y ago

I don't know if it's impressive or worrying regarding how much Posit/RStudio as a company have completely taken over a programming language. I have colleagues who write R code all day, and they don't know the difference between RStudio the IDE and R the language. They can't perform basic tasks such as defining a function, or adding a column without the tidyverse.

fbdab103 4y ago
I see that same thing all the time. RStudio = R, and they have no concept of the language without the IDE. Which is fine, I guess? They are not professional programmers, but people with a job to do. It is just frustrating to me when they RShiny all the things, and I have to pickup the pieces because the spaghetti has collapsed under its own weight.
- peatmoss 4y ago
  I see it a little bit more like a meritocracy in that space that RStudio has largely won. Microsoft has certainly made an attempt to become the face of R. Remember when they purchased Revolution Analytics and started baking it into everything? Or how about Oracle making a play on Graalvm with "FastR"? RStudio has somewhat indefatigably looked for pain points that R users had back in the olden days. And yes, I remember R back when lattice plots were the hottest game in town. Not only has the Tidyverse made R infinitely more pleasant to work with, the community around Tidyverse has made the R community infinitely less insufferable for newcomers. The obtuse responses to beginners on the old R mailing lists were legendary. My preferred R IDE is emacs, and I'm not upset with what RStudio has done. When I interact with devtools, I don't feel that I'm wanting for functionality. This is in spite of the fact that RStudio has heavily integrated devtools functionality into their IDE, and could arguably have made devtools less easy to work with using generic tooling. I understand the mediocrity of many RShiny applications in terms of how they are written. But, another way of looking at all those dashboards and interactive applications, is that they are AMAZING prototypes written by scientists, grad students, journalists, etc., who would have never been able to build them some other way. If we forced everyone to write these apps in Rust, of course the average quality would be higher, but we'd also have way fewer of them. Democratization of technology means that its users will be more... medium.