4 ms·

The pandas api is a mess and that's why it feels that way. It is a great tool, and very powerful, but boy does it make the user's life difficult with a pretty c

The pandas api is a mess and that's why it feels that way. It is a great tool, and very powerful, but boy does it make the user's life difficult with a pretty convoluted API that makes it very hard to automatically discover functionality. As others have said, unless you use it all the time, you're essentially in for a bad time that involves lots of Googling and SO browsing even to get basic things done in pandas. I say this as someone who had developed a few smaller projects that utilize pandas extensively, so I'm not just criticizing it without having used it.

EForEndeavour 7y ago
As a daily pandas user for a couple years, I agree that the API is rough in places, though the official docs are extremely helpful. Also, the Stack Overflow community has come up with some helpful "missing manual" writeups, such as "Pandas Merging 101" https://stackoverflow.com/questions/53645882/pandas-merging-101 https://stackoverflow.com/questions/53645882/pandas-merging-... API pain points off the top of my head: - There are both `pivot` and `pivot_table` methods which behave slightly differently. I'd have to look up why `pivot` exists; I've learned simply to reach for `pivot_table` every time, since it aggregates more flexibly. - The dreaded SettingWithCopyWarning is a huge pain for new pandas users, and simply should not exist in any mature data analysis software. - Unlike in SQL, in pandas you can take a data table (a DataFrame) and group it by one or more columns without immediately specifying an aggregation function. The result of this `groupby` operation is a "groupby object", which contains all the information of the input DataFrame, but behaves completely differently from a DataFrame. By contrast, in R's tidyverse, my understanding is that when you group a tibble (the tidyverse's data table), the result is still a tibble, just grouped. - It took a while to mentally sort out the similarities and differences of merge, join, append, and concatenate. In practice, I almost always reach for `merge` or `concat`.

EForEndeavour 7y ago

As a daily pandas user for a couple years, I agree that the API is rough in places, though the official docs are extremely helpful. Also, the Stack Overflow community has come up with some helpful "missing manual" writeups, such as "Pandas Merging 101" https://stackoverflow.com/questions/53645882/pandas-merging-101 https://stackoverflow.com/questions/53645882/pandas-merging-... API pain points off the top of my head: - There are both `pivot` and `pivot_table` methods which behave slightly differently. I'd have to look up why `pivot` exists; I've learned simply to reach for `pivot_table` every time, since it aggregates more flexibly. - The dreaded SettingWithCopyWarning is a huge pain for new pandas users, and simply should not exist in any mature data analysis software. - Unlike in SQL, in pandas you can take a data table (a DataFrame) and group it by one or more columns without immediately specifying an aggregation function. The result of this `groupby` operation is a "groupby object", which contains all the information of the input DataFrame, but behaves completely differently from a DataFrame. By contrast, in R's tidyverse, my understanding is that when you group a tibble (the tidyverse's data table), the result is still a tibble, just grouped. - It took a while to mentally sort out the similarities and differences of merge, join, append, and concatenate. In practice, I almost always reach for `merge` or `concat`.