9 ms·

What should one do if after following those million guides it still doesn’t stick? I always end up googling what I want and hit SO. Is there something wrong wit

What should one do if after following those million guides it still doesn’t stick? I always end up googling what I want and hit SO. Is there something wrong with me or does numpy and pandas seem more difficult than they should be?

lawlorino 7y ago
I really wouldn't worry about it - I've been using Python for data work for the last 5 years or so and I have to look stuff all the time. Eventually the basic stuff sticks but it's like any kind of coding, I don't think anyone ever hits the point where they hardly ever have to Google stuff. One useful tip I can suggest is to create a repo for useful code snippets, so if you ever find yourself doing something new that you think you might need again, just spend a bit of time commenting and describing it and add it to the repo. That way instead of having to spend time searching you'll hopefully remember doing it before and be able to find it easily.
- xapata 7y ago
  Back in the old days, experts collected reference manuals. Googling things is the same as flipping through the pages of a manual.
- Foivos 7y ago
  After googling the same things every other week, I started using a snippets application (in my case it is SnippetsLab) where I wrote a nice description and keywords for every snippet. Life is so much easier that way.
BeetleB 7y ago
>Is there something wrong with me or does numpy and pandas seem more difficult than they should be? If you're only an occasional user, this will be your life forever. My experience with pandas is that if you use it heavily for 3 months, then things start to "stick" and you need to look it up less often. Unfortunately I changed jobs and have forgotten most of pandas, so I'm back to looking things up again.
- disgruntledphd2 7y ago
  pandas has a terrible API (IMO obviously). It looks like a weird mixture of Numpy and pre-2010 R code. Like it has great fucntionality, but I waste so much time trying to figure out how to do something with pandas. And I can remember the R-API's, which are even more annoying ;)
  - luckydata 7y ago
    re: weird mixture of Numpy and pre-2010R It does because it is. Pandas was written by someone that was just starting out with Python at the time and was coming from programming in R.
    - EForEndeavour 7y ago
      Pandas was extensively rewritten since its first few versions, and its GitHub repo has 1500+ contributors now, so I doubt that Wes McKinney's initial lack of python experience had much to do with the API rough spots of modern pandas. "Pre-2010 R" doesn't mean much to me, but I can confidently point out that pandas "looks like a weird mixture of numpy" because it is based heavily on numpy.
      - disgruntledphd2 7y ago
        It basically means pre-tidyverse R. pandas looks like a weird hybrid of Numpy and R. It's such a shame, as python has marvellous abstraction facilities, and yet they aren't used in pandas (__methods__ etc).
_coveredInBees 7y ago
The pandas api is a mess and that's why it feels that way. It is a great tool, and very powerful, but boy does it make the user's life difficult with a pretty convoluted API that makes it very hard to automatically discover functionality. As others have said, unless you use it all the time, you're essentially in for a bad time that involves lots of Googling and SO browsing even to get basic things done in pandas. I say this as someone who had developed a few smaller projects that utilize pandas extensively, so I'm not just criticizing it without having used it.
- EForEndeavour 7y ago
  As a daily pandas user for a couple years, I agree that the API is rough in places, though the official docs are extremely helpful. Also, the Stack Overflow community has come up with some helpful "missing manual" writeups, such as "Pandas Merging 101" https://stackoverflow.com/questions/53645882/pandas-merging-101 https://stackoverflow.com/questions/53645882/pandas-merging-... API pain points off the top of my head: - There are both `pivot` and `pivot_table` methods which behave slightly differently. I'd have to look up why `pivot` exists; I've learned simply to reach for `pivot_table` every time, since it aggregates more flexibly. - The dreaded SettingWithCopyWarning is a huge pain for new pandas users, and simply should not exist in any mature data analysis software. - Unlike in SQL, in pandas you can take a data table (a DataFrame) and group it by one or more columns without immediately specifying an aggregation function. The result of this `groupby` operation is a "groupby object", which contains all the information of the input DataFrame, but behaves completely differently from a DataFrame. By contrast, in R's tidyverse, my understanding is that when you group a tibble (the tidyverse's data table), the result is still a tibble, just grouped. - It took a while to mentally sort out the similarities and differences of merge, join, append, and concatenate. In practice, I almost always reach for `merge` or `concat`.
squaresmile 7y ago
I suggest looking into matplotlib structures: figures and axes. I think this article [0] is pretty good at detailing how to work with them. They definitely can be confusing but I think most can grasp what to use after reading the article. [0] http://jonathansoma.com/lede/algorithms-2017/classes/fuzziness-matplotlib/how-pandas-uses-matplotlib-plus-figures-axes-and-subplots/ http://jonathansoma.com/lede/algorithms-2017/classes/fuzzine...
Foivos 7y ago
It is possible to do the same things in Pandas is many different ways. It is good to have this flexibility, but it is confusing for new users. On top of that the bracket operator ([) is overloaded in many ways. Things started to make sense after I read a very good book on Pandas[1]. Reading a book is better than reading blog posts, because it is consistent. In contrast, reading small tutorials for every little thing is confusing, because every blog post is using a different way to do the same thing. [1] https://github.com/jakevdp/PythonDataScienceHandbook https://github.com/jakevdp/PythonDataScienceHandbook

lawlorino 7y ago

I really wouldn't worry about it - I've been using Python for data work for the last 5 years or so and I have to look stuff all the time. Eventually the basic stuff sticks but it's like any kind of coding, I don't think anyone ever hits the point where they hardly ever have to Google stuff. One useful tip I can suggest is to create a repo for useful code snippets, so if you ever find yourself doing something new that you think you might need again, just spend a bit of time commenting and describing it and add it to the repo. That way instead of having to spend time searching you'll hopefully remember doing it before and be able to find it easily.

xapata 7y ago
Back in the old days, experts collected reference manuals. Googling things is the same as flipping through the pages of a manual.
Foivos 7y ago
After googling the same things every other week, I started using a snippets application (in my case it is SnippetsLab) where I wrote a nice description and keywords for every snippet. Life is so much easier that way.

BeetleB 7y ago

>Is there something wrong with me or does numpy and pandas seem more difficult than they should be? If you're only an occasional user, this will be your life forever. My experience with pandas is that if you use it heavily for 3 months, then things start to "stick" and you need to look it up less often. Unfortunately I changed jobs and have forgotten most of pandas, so I'm back to looking things up again.

disgruntledphd2 7y ago
pandas has a terrible API (IMO obviously). It looks like a weird mixture of Numpy and pre-2010 R code. Like it has great fucntionality, but I waste so much time trying to figure out how to do something with pandas. And I can remember the R-API's, which are even more annoying ;)
- luckydata 7y ago
  re: weird mixture of Numpy and pre-2010R It does because it is. Pandas was written by someone that was just starting out with Python at the time and was coming from programming in R.
  - EForEndeavour 7y ago
    Pandas was extensively rewritten since its first few versions, and its GitHub repo has 1500+ contributors now, so I doubt that Wes McKinney's initial lack of python experience had much to do with the API rough spots of modern pandas. "Pre-2010 R" doesn't mean much to me, but I can confidently point out that pandas "looks like a weird mixture of numpy" because it is based heavily on numpy.
    - disgruntledphd2 7y ago
      It basically means pre-tidyverse R. pandas looks like a weird hybrid of Numpy and R. It's such a shame, as python has marvellous abstraction facilities, and yet they aren't used in pandas (__methods__ etc).

_coveredInBees 7y ago

The pandas api is a mess and that's why it feels that way. It is a great tool, and very powerful, but boy does it make the user's life difficult with a pretty convoluted API that makes it very hard to automatically discover functionality. As others have said, unless you use it all the time, you're essentially in for a bad time that involves lots of Googling and SO browsing even to get basic things done in pandas. I say this as someone who had developed a few smaller projects that utilize pandas extensively, so I'm not just criticizing it without having used it.

EForEndeavour 7y ago
As a daily pandas user for a couple years, I agree that the API is rough in places, though the official docs are extremely helpful. Also, the Stack Overflow community has come up with some helpful "missing manual" writeups, such as "Pandas Merging 101" https://stackoverflow.com/questions/53645882/pandas-merging-101 https://stackoverflow.com/questions/53645882/pandas-merging-... API pain points off the top of my head: - There are both `pivot` and `pivot_table` methods which behave slightly differently. I'd have to look up why `pivot` exists; I've learned simply to reach for `pivot_table` every time, since it aggregates more flexibly. - The dreaded SettingWithCopyWarning is a huge pain for new pandas users, and simply should not exist in any mature data analysis software. - Unlike in SQL, in pandas you can take a data table (a DataFrame) and group it by one or more columns without immediately specifying an aggregation function. The result of this `groupby` operation is a "groupby object", which contains all the information of the input DataFrame, but behaves completely differently from a DataFrame. By contrast, in R's tidyverse, my understanding is that when you group a tibble (the tidyverse's data table), the result is still a tibble, just grouped. - It took a while to mentally sort out the similarities and differences of merge, join, append, and concatenate. In practice, I almost always reach for `merge` or `concat`.

squaresmile 7y ago

I suggest looking into matplotlib structures: figures and axes. I think this article [0] is pretty good at detailing how to work with them. They definitely can be confusing but I think most can grasp what to use after reading the article. [0] http://jonathansoma.com/lede/algorithms-2017/classes/fuzziness-matplotlib/how-pandas-uses-matplotlib-plus-figures-axes-and-subplots/ http://jonathansoma.com/lede/algorithms-2017/classes/fuzzine...

Foivos 7y ago

It is possible to do the same things in Pandas is many different ways. It is good to have this flexibility, but it is confusing for new users. On top of that the bracket operator ([) is overloaded in many ways. Things started to make sense after I read a very good book on Pandas[1]. Reading a book is better than reading blog posts, because it is consistent. In contrast, reading small tutorials for every little thing is confusing, because every blog post is using a different way to do the same thing. [1] https://github.com/jakevdp/PythonDataScienceHandbook https://github.com/jakevdp/PythonDataScienceHandbook