9 ms·
Pandas 3.0
- optimalsolver 9mo agoHow soon will the leading LLMs ingest the updated documentation? Because I'm certainly not going to.
- uncletoxa 9mo agoUse context7 mcp. It'll do the trick
- leadingthenet 9mo agoI've been sleeping on this, works like a charm!
- OutOfHere 9mo agoIn my experience, it would take a year to ingest it natively, and two years to also ingest enough coding examples.
- swyx 9mo agoits not even about the ingest, every major semver change now is a problem because now LLMs will need to contextually distinguish whether or not they are expected to output Pandas 2 or 3, unless ofc you explicitly prompt it.
- OutOfHere 9mo agoI wouldn't worry about it because over a longer period, this automatically leans toward the more recent versions. There are multiple forces that exist to make this happen. The main exception is for legacy code requiring maintenance when they are unwilling to upgrade Pandas.
- swyx 9mo agoyes but a lot of legacy code wil still need to be maintained and written. you dont see how this can be confusing/annoying?
- g-mork 9mo agoThis is the most misunderstood aspect of how marketing has changed recently
- esafak 9mo agoYou could create skills out of the docs if you use it a lot. https://agentskills.io/ https://agentskills.io/
- postalcoder 9mo agoI've migrated off of pandas to polars for my workflows to reap the benefit of, in my experience a 10-20x speedup on average. I can't imagine anything bringing me back short of a performance miracle. LLMs have made syntax almost a non-barrier.
- alex7o 9mo agoSame, also polars works on typescript which I used at some point out move my data from backend to frontend
- gHA5 9mo agoDo you not experience LLM generated code constantly trying to use Pandas' methods/syntax for Polars objects?
- postalcoder 9mo agoThere were some growing pains in gpt-3.5 to gpt-4 era, but not nowadays (shoutout to the now-defunct Phind, which was a game changer back then).
- crimsoneer 9mo agoThe fact they pivoted away from their very compelling core offering (AI stack overflow) to complete with loveable etc in the "AI generated apps" giant fight continues to baffle me. Though I guess model updates ate their lunch.
- postalcoder 9mo agoMy guess is that their pivot came after distress, and was not the cause of it. It'd be great to have @rushingcreek write a post-mortem. I think it'd benefit a lot of people because I honestly don't have a monday morning playbook of what could have saved them. Like you said, perhaps the demise of phind was inevitable, with large models displacing them kind of like how Spotify displaced music piracy.
- 9mo ago
- OutOfHere 9mo agos/impactfull/impactful
- Havoc 9mo agoRegex is great when one is communicating with machines
- edschofield 9mo agoThe design of Pandas is inferior in every way to Polars: API, memory use, speed, expressiveness. Pandas has been strictly worse since late 2023 and will never close the gap. Polars is multithreaded by default, written in a low-level language, has a powerful query engine, supports lazy, out-of memory execution, and isn’t constrained by any compatibility concerns with a warty, eager-only API and pre-Arrow data types that aren’t nullable. It’s probably not worth incurring the pain of a compatibility-breaking Pandas upgrade. Switch to Polars instead for new projects and you won’t look back.
- v3ss0n 9mo agoSounds too much like an advertisement. Also we need to watch out when diving into Polars . Polars is VC backed Opensource project with cloud offering , which may become an opencore project - we know how those goes.
- gkbrk 9mo ago> we know how those go They get forked and stay open source? At least this is what happens to all the popular ones. You can't really un-open-source a project if users want to keep it open-source.
- stingraycharles 9mo agoDepends on your definition of popular; plenty of examples where the business interests don't align well with open source.
- v3ss0n 9mo agonot many can maintain a complex project in full time.
- quentindanjou 9mo agoI was also thinking that this comment looks like an AD. Pandas does not have any paid option and isn't made directly for profit.
- jtrueb 9mo agoThat timestamp resolution discrepancy is going to cause so many problems
- EForEndeavour 9mo agoDo you mean the new default datetime resolution of microseconds instead of the previous nanosecond resolution? Obviously this will require adjustments to any code that requires ns resolution, but I'd bet that's a tiny minority of all pandas code ever written. Do you have a particular use case in mind for the problems this will cause?
- jtrueb 9mo agoI would describe it as the huge majority, reflecting on my pandas use over the years. Pretty much all of the data worth exploring in pandas over excel, some data gui, or polars involves timestamps.
- alexcasalboni 9mo agoHaven't used pandas in a while, but Copy-on-Write sounds pretty cool! Is there any public benchmark I can check in 2026?
- teekert 9mo agoI have deep respect for Pandas, it, and Jupyter-lab were my intro to programming. And it worked much better for me, I did some "intro to Python" courses, but it was all about strs and ints. And yes, you can add strs together! Wow magic... Not for me. For me it all clicked when I first looped through a pile of Excel files (pd.read_excel()), extracted info I needed and wrote a new Excel file... Mind blown. From there, of course, you slowly start to learn about types etc, and slowly you start to appreciate libraries and IDEs. But I knew tables, and statistics and graphs, and Pandas (with the visual style of Notebooks) lead me to programming via that familiar world. At first with some frustration about Pandas and needing to write to Excel, do stuff, and read again, but quickly moving into the opposite flow, where Excel itself became the limiting factor and being annoyed when having to use it. I offered some "Programming for Biologists" courses, to teach people like me to do programming in this way, because it would be much less "dry" (pd.read.excel().barplot() and now you're programming). So far, wherever I offered the courses they said they prefer to teach programming "from the base up". Ah well! I've been told I'm not a programmer, I don't care. I solve problems (and that is the only way I am motivated enough to learn, I can't sit down solving LeetCode problems for hours, building exactly nothing). (To be clear, I now do the Git, the Vim, the CI/CD, the LLM, the Bash, The Linux, the Nix, the Containers... Just like a real programmer, my journey was just different, and suited me well, I believe others can repeat my journey and find joy in programming, via a different route.)
- QuadmasterXLII 9mo agoUgh, I'm still recovering from numpy breaking changes with 2.0
- kayson 9mo agoAre there any pandas alternatives that offer stronger column typing? Ideally something where I can have the schema defined in advance, validate the data, then have the type checker be smart enough to know that df.foo exists and is float and df.bar doesn't. I tried pandera and it left a lot to be desired. Static frame [1] seems promising but doesn't appear to be popular for some reason. 1. https://static-frame.readthedocs.io/en/latest/ https://static-frame.readthedocs.io/en/latest/
- gku 9mo agoThe need to upgrade Pandas, combined with emerging AI tools, might accelerate Polars adoption, let’s see what happens.