3 ms·

I am experienced with pandas and I understand its uses - but it definitely makes learning it much more confusing for new people - like the first time you see a

I am experienced with pandas and I understand its uses - but it definitely makes learning it much more confusing for new people - like the first time you see a multi-index you're just like 'OMG what'. It also feels like it silghtly breaks the mental model of a dataframe for me - like why am I treating these columns as special all of a sudden? Sometimes that can make things slicker, but 90% of the time for me it just necessitates the need for .reset_index or index=False or equivalent. If I want to use index to optimise something - that should be a deliberate act, not something pushed on me by the API.

cardosof 5y ago
As someone with years of working experience with dplyr who had to learn pandas, 100% agree. And thanks for that post, I thought it was just me.
tgb 5y ago
I think it helps coming from another direction where dataframes are fancy 2D numpy arrays or a fancy 'dict-of-dicts'. I do like being able to query from index with .loc. Nonetheless I basically agree with you.
lysecret 5y ago
yea I know what you mean. The main use I have found is something like. Say you have a price per company_id and per transaction_id. Now you have some function, which for the sake of it, takes it to an exponent. i used to write a lot of functions which went like this: def convert_price(price_frame:pd.DataFrame) -> pd.DataFrame: price_frame["new_price"] = np.exp(price_frame) return price_frame and i came back to these functions, and i was always like. hmm what needs to be in the price frame. which columns etc. Also it mutates the state of the data frame while i now write functions like def convert_price(price:pd.Series) -> pd.Series: return np.exp(price) However what is good as well you can treat the series as a single dimensional array and do operations on it. Its not a perfect example since im not using the ids haha but you might see what i mean :D

cardosof 5y ago

As someone with years of working experience with dplyr who had to learn pandas, 100% agree. And thanks for that post, I thought it was just me.

tgb 5y ago

I think it helps coming from another direction where dataframes are fancy 2D numpy arrays or a fancy 'dict-of-dicts'. I do like being able to query from index with .loc. Nonetheless I basically agree with you.

lysecret 5y ago

yea I know what you mean. The main use I have found is something like. Say you have a price per company_id and per transaction_id. Now you have some function, which for the sake of it, takes it to an exponent. i used to write a lot of functions which went like this: def convert_price(price_frame:pd.DataFrame) -> pd.DataFrame: price_frame["new_price"] = np.exp(price_frame) return price_frame and i came back to these functions, and i was always like. hmm what needs to be in the price frame. which columns etc. Also it mutates the state of the data frame while i now write functions like def convert_price(price:pd.Series) -> pd.Series: return np.exp(price) However what is good as well you can treat the series as a single dimensional array and do operations on it. Its not a perfect example since im not using the ids haha but you might see what i mean :D