4 ms·
Sounds like we might be coming from different perspectives. I honestly don't use any DF libraries often, and really only Pandas. I used to use pandas a fair amo
by godelski 3y ago
Sounds like we might be coming from different perspectives. I honestly don't use any DF libraries often, and really only Pandas. I used to use pandas a fair amount, but that was years ago, and now I only have to reach for it a few times a year. So maybe the docs are good for people that already have deeper experience. Because I think just the fact that you have used datafusion and duckdb illustrates that you're more skilled in this domain than I am, because I haven't used those haha.
But I do think making good docs is quite hard. You usually have multiple audiences that you might not even be aware of. Which makes one of the most important things to do is keep an open ear to listen for them. It's easy to get trapped thinking you got your audience but you're actually closing the door to many more groups (unintentionally). It's also just easy to be focused on the "real" work and not think about docs.
- sanderjd 3y agoWhat, specifically, is bad about the docs? This whole thread is people who just looked at the home page, saw that it is "DataFrames", but didn't know what that means and came here to complain. Nobody has said anything about issues with the docs for someone who understands what a data frame is (or spent like two minutes looking that up) but is struggling to figure out how to use this library specifically.
- godelski 3y agoI think your experience is probably making it difficult to understand the noob side of things. For me, I've struggled with simply slicing up a dataframe. And as I specified, these aren't tools I use a lot, so the "who understands what a data frame is" probably doesn't apply to me very well and we certainly don't need the pejorative nature suggesting that it is trivially understood or something I should know through divine intervention. I'm sure it's not difficult, but it can take time for things to click. Hell, I can do pretty complex integrals and derivatives and now so much of that seems trivial to me now but I did struggle when learning it. Don't shame people for not already knowing things when they are explicitly trying to learn things. Shame the people that think they know and refuse to learn. There's no reason to not be nice. Having done a lot of teaching I have a note, don't expect noobs to be able to articulate their problems well. They're noobs. They have the capacity to complain but it takes expertise to have clarify that complaint, turning it into a critique. I get that this is frustrating, but being nice turns noobs into experts and often friends too.
- sanderjd 3y agoI really think this is a misunderstanding of the purpose of different kinds of documentation. The documentation of a new tool for a mature technique is just not the primary place to focus on writing a beginners' tutorial / course on using that technique. Certainly, "the more the merrier" is a good mantra for documentation, so if they do add such material, all the better. But it is very sensible for it to not be the focus. The focus should be, "how can you use this specific iteration of a tool for this technique to do the things you already know how to do". Nobody is suggesting that you should be an expert on data frames "through divine intervention". But the place to expect to learn about those things is the many articles, tutorials, courses, and books on the subject, not the website of one specific new tool in the space. If you're really interested in learning about this, a fairly canonical place to start would be "Python for Data Analysis"[0] by Wes McKinney, the creator of pandas and one of the creators of the arrow in-memory columnar data format that most of these projects build atop now. This is a (multiple-) book length topic, not a project landing page length topic. 0: https://wesmckinney.com/book/ https://wesmckinney.com/book/
- godelski 3y ago> But it is very sensible for it to not be the focus. Sure. I mean devs can do whatever they want. But the package is a few years old now and they do frequently advertise, so I don't think it makes make it more approachable for... you know... noobs. This is a bit difficult of a conversation too, because you've moved the goal post. I've always held the context of noob, but now you've shifted to just be dismissive of noobs. Totally fine, but different from the last comment. > But the place to expect to learn about those things is the many articles, tutorials, courses, and books on the subject, not the website of one specific new tool in the space. I actually disagree. This is the outsourcing I expressed previously, but it's clear from the number of complaints that this is not sufficient for a novice. You do seem passionate about this issue, and so maybe you have the opportunity to fill that gap. But I very much think that official documentation is supposed to be the best place. Frankly because it is written by the people who have a full understanding of the system and how it all integrates together. I'm sure you've run into tons of Medium tutorials that get the job done but are also utter garbage and misinform users. It isn't surprising when most of these are written by those in the process of learning, and are better than nothing, but they are entirely insufficient. The whole point of creating a library is to save people time. That time includes onboarding. For example of good docs, I highly recommend the vim docs. Even man pages are often surprisingly good.
- leksak 3y agoI can't speak for the Python side of the Polars docs but coming from Python and Pandas to Rust and Polars hasn't always been easy. To be fair, that isn't just about docs but also finding articles or Stack Overflow answers for people doing similar things.
- sanderjd 3y agoThat certainly makes sense!
- thecodedmessage 3y agoThe Rust docs are for some reason much worse than the Python docs, or at least that used to be the case
- anjanb 3y agoI'm a dataframes noob. I saw this post and the performance claims attracted me. I went to chatGPT to understand what dataframes were about. Then on udemy, I searched for a polar course. A course required pre-requisites : a bit about jupyter notebooks and pandas. Then I went through a few modules of a pandas course. Now, I'm going through a polars course. Altogether, I spent about 2-3 hours to setup the environment and know what this is all about. A little bit context would have helped to have attracted a lot more noobs.g
- _dain_ 3y ago"How To Be A Pandas Expert"[1] is a good primer on dataframes. There's a certain mental model you need to use dataframes effectively but it's not apparent from reading the official docs. The video makes it explicit: dataframes are about like-indexed one-dimensional data, and every dataframe operation can be understood in terms of what it does to the index. [1] https://www.youtube.com/watch?v=oazUQPrs8nw https://www.youtube.com/watch?v=oazUQPrs8nw
- sanderjd 3y agoYour first paragraph makes perfect sense! I was nodding along. But then your concluding sentence was a bit of a record scratch for me. This all worked as intended! You knew what the project was about - "data frames" - and what might make it attractive to you - the performance claims - and then you went and followed exactly the right path to get the context you needed to understand what's going on with it. It's a big topic that you were able to spin up on to a basic level in 2-3 hours, by pulling on strings starting at this landing page. This is a very successful outcome. I'd also recommend this book: https://wesmckinney.com/book/ https://wesmckinney.com/book/. It's not about polars, but you'd be able to transfer its ideas to polars easily once you read it.