3 ms·

Does that work for chained pandas expressions too? I love writing chained expressions but debugging them is like visiting a special kind of hell.

iruoy 4y ago
> This highlighting will occur for every frame in the traceback. For instance, if a similar error is part of a complex function call chain, the traceback would display the code associated to the current instruction in every frame: Traceback (most recent call last): File "test.py", line 14, in <module> lel3(x) ^^^^^^^ File "test.py", line 12, in lel3 return lel2(x) / 23 ^^^^^^^ File "test.py", line 9, in lel2 return 25 + lel(x) + lel(x) ^^^^^^ File "test.py", line 6, in lel return 1 + foo(a,b,c=x['z']['x']['y']['z']['y'], d=e) ~~~~~~~~~~~~~~~~^^^^^ TypeError: 'NoneType' object is not subscriptable
hnews_account_1 4y ago
All my data devs please stand up! I sometimes sacrifice readability just because I hate creating variables. But then if it affects debugging times, my boss would be furious. As such, I use a full debugger anyway so I can trace quickly.
emehex 4y ago
If you like writing chain-able pandas, you should check out: https://github.com/maxhumber/redframes https://github.com/maxhumber/redframes
- IanOzsvald 4y ago
  What's your plan for redframes? It looks really new? I'm the co-author of O'Reilly's High Performance Python so I'm always on the lookout for pandas alternatives. Are you looking at speed implications too? Bigger-than-RAM use cases? "Easier than pandas" only (which, of course, is totally huge in its own right)?
  - emehex 4y ago
    "Easier than pandas" is the most important consideration for right now! If pandas is "Jira", I consider redframes to be "Trello" (appropriate for most, but not all use cases). I actually was inspired to build the library after teaching a one week intensive pandas course to a couple of Data Scientists @ a Fortune 500... (pandas is really hard for beginners!) While performance and OOM aren't priorities right now, I'd love to one day replace the pandas "backend" with Arrow (or something else) once I nail the API :)
  - bbkane 4y ago
    Ian, what are your thoughts on https://www.pola.rs/ https://www.pola.rs/ ? I've used it for a few CSV tasks and I really liked the API
    - IanOzsvald 4y ago
      I interviewed author Ritchie Vink on my newsletter (NotANumber) some months back, he's smart and the library has a nice design. I still barely know anyone trying it, it did get a write up just recently here: https://news.ycombinator.com/item?id=32998040 https://news.ycombinator.com/item?id=32998040 On one of my recent Higher Performance training courses a hedge fund person said they'd tried it with mixed success - some things faster than Pandas, others more expensive on RAM. I'm hoping to have a play soon but have only lightly tinkered so far. The parallel-by-design nature is nice, but I think the API is still evolving rapidly making it harder to develop with.

iruoy 4y ago

> This highlighting will occur for every frame in the traceback. For instance, if a similar error is part of a complex function call chain, the traceback would display the code associated to the current instruction in every frame: Traceback (most recent call last): File "test.py", line 14, in <module> lel3(x) ^^^^^^^ File "test.py", line 12, in lel3 return lel2(x) / 23 ^^^^^^^ File "test.py", line 9, in lel2 return 25 + lel(x) + lel(x) ^^^^^^ File "test.py", line 6, in lel return 1 + foo(a,b,c=x['z']['x']['y']['z']['y'], d=e) ~~~~~~~~~~~~~~~~^^^^^ TypeError: 'NoneType' object is not subscriptable

hnews_account_1 4y ago

All my data devs please stand up! I sometimes sacrifice readability just because I hate creating variables. But then if it affects debugging times, my boss would be furious. As such, I use a full debugger anyway so I can trace quickly.

emehex 4y ago

If you like writing chain-able pandas, you should check out: https://github.com/maxhumber/redframes https://github.com/maxhumber/redframes

IanOzsvald 4y ago
What's your plan for redframes? It looks really new? I'm the co-author of O'Reilly's High Performance Python so I'm always on the lookout for pandas alternatives. Are you looking at speed implications too? Bigger-than-RAM use cases? "Easier than pandas" only (which, of course, is totally huge in its own right)?
- emehex 4y ago
  "Easier than pandas" is the most important consideration for right now! If pandas is "Jira", I consider redframes to be "Trello" (appropriate for most, but not all use cases). I actually was inspired to build the library after teaching a one week intensive pandas course to a couple of Data Scientists @ a Fortune 500... (pandas is really hard for beginners!) While performance and OOM aren't priorities right now, I'd love to one day replace the pandas "backend" with Arrow (or something else) once I nail the API :)
- bbkane 4y ago
  Ian, what are your thoughts on https://www.pola.rs/ https://www.pola.rs/ ? I've used it for a few CSV tasks and I really liked the API
  - IanOzsvald 4y ago
    I interviewed author Ritchie Vink on my newsletter (NotANumber) some months back, he's smart and the library has a nice design. I still barely know anyone trying it, it did get a write up just recently here: https://news.ycombinator.com/item?id=32998040 https://news.ycombinator.com/item?id=32998040 On one of my recent Higher Performance training courses a hedge fund person said they'd tried it with mixed success - some things faster than Pandas, others more expensive on RAM. I'm hoping to have a play soon but have only lightly tinkered so far. The parallel-by-design nature is nice, but I think the API is still evolving rapidly making it harder to develop with.