4 ms·

pandas

• mynameis_jeff 4y ago
  would give https://github.com/modin-project/modin https://github.com/modin-project/modin a shot
• SnooSux 4y ago
  It's been done: https://github.com/pola-rs/polars https://github.com/pola-rs/polars But I'm sure there's always room for improvement
  • rytill 4y ago
    It’s not like polars is a drop-in replacement, it has a totally different API.
    • mrtranscendence 4y ago
      You wrote "it has a totally different API", did you mean "it has an actually sane API?" Because that's what I think of when I compare pandas to polars.
      • ies7 4y ago
        Not if you've legacy apps using pandas everywhere. Same API means I can import polars as pd and be done with it.
• fgh 4y ago
  The answer would then be to have a look at polars.
• w-m 4y ago
  This is a curious reply for me. I would think that there are very few parts in pandas that could be sped-up by reimplementing them with a compiled language. Pandas is plenty fast for the built-in methods, it only gets slow when you start interfacing with Python, e.g. by doing an `.apply` with your custom Python method. Obviously this interfacing part is impossible to speed up by reimplementing parts of pandas (you'd need a different API instead).
• curiousgal 4y ago
  I remember, when trying to squeeze some performance out of it, that a lot of the overhead came from it trying to infer types.