brahbrah

6 ms·

31.
by brahbrah 4y ago
Not necessarily true. I have seen in the past employees be “laid off” solely due to their performance. Rather than fire the employee for performance reasons they opt to eliminate the role altogether, and not rehire for it. The team had a lo
32.
by brahbrah 4y ago
Just want to point out that polars is not a complete replacement for pandas and they will likely live side by side in the data ecosystem. One of the main and original intentions of pandas is replacing highly dynamic and changing excel model
33.
by brahbrah 4y ago
I know it’s not the focus of the article, but a personal pet-peeve of mine is when pandas gets mentioned in articles and discussions regarding alternatives to sql and data pipeline tools. Pandas was never meant to be a replacement for sql,
34.
by brahbrah 4y ago
Check out hex’s reactive notebooks https://hex.tech/blog/hex-two-point-oh/
35.
by brahbrah 4y ago
Sometimes yes and sometimes no. For example if you had price data and you wanted to bump prices up a dollar in 2020 how would you do that in sql? In pandas it’s: prices_df.loc['2020'] += 1 If you had a temperature forecas
36.
by brahbrah 4y ago
Pandas is in a separate category from all of these, including polars. If you were to say “pandas in long format only” then yes that would be correct, but the power of pandas comes in its ability to work in a long relational or wide ndarray
37.
by brahbrah 4y ago
So in pandas you might get that warning when you’re trying to do an in place operation like so: df.loc['2023-01-01': '2023-12-31'] *= 2 This doubles every row in 2023 in your frame. In polars there’s no mutating
38.
by brahbrah 4y ago
If you have a dataframe of power plant capacities and a dataframe of power plant capacity reductions how would you figure out the available capacities of all your power plants. In pandas it would be `capacities_df - reductions_df`. How woul
39.
by brahbrah 4y ago
I'm not saying that polars isn't faster. In fact in my other comment here I mention that polars is much better than pandas at what polars does (it's not a drop in replacement). I'm just saying that most of the times (not
40.
by brahbrah 4y ago
I have the same dynamic at my job. It’s a classic case of they can’t do what I can do and I can’t do what they can so let’s work together. It’s painful but necessary. I feel like it’s a perfectly valid excuse though. They have training in s
41.
by brahbrah 4y ago
By any chance were you iterating over your pandas dataframe or using .apply? I’d be surprised by any properly formatted (i.e. vectorized) pandas operation that takes that long for data that fits in memory
42.
by brahbrah 4y ago
I like polars a lot. It’s better than pandas at what it does. But it only accounts for a subset of functionality that pandas does. Indexes are not just some implementation details of dataframes. They are fundamental to the representation of