3 ms·
My personal anectodal experience : 3 weeks ago, I had a new pet project, and decided to go with Python again, which I left many years ago, before Python typing
by antpls 6y ago
My personal anectodal experience :
3 weeks ago, I had a new pet project, and decided to go with Python again, which I left many years ago, before Python typing was a thing.
I didn't know where to start, so I started by creating a file with data models I would need. Basically C struct-like / record models. All the classes inherited NamedTuple, and I typed all the attributes. I then created a few pure functions based on those classes. NewType is a great addition. When you work with many int or float variables representing different things in real life (price, amount, quantities, etc), it makes the code cleaner and catch some mistakes when you are tired at the end of the day.
I gradually stopped to type things when I introduced pandas into the project and types started to be way more abstract. There is probably a way to make typing work with pandas, but I wanted to learn pandas first and get the project running, so I didn't really try. I expected pandas to do dynamic things under the hood to work, so I didn't bother to understand how types work with it.
Of course, being a hobby project, there is no unit test and no documentation, so I may not see all the pros and cons of typing. But I have a feeling gradual typing is an interesting idea that will continue to improve.
It's nice to have the support of static typing when you need to, but also be able to "just hack" a solution real quick when needed.
- matesz 6y ago> there is probably a way to make typing work with pandas So we did use a lot of pandas as well. The way to make it work is to create custom types and of course each dataframe will have it's own type which is going to be a total mess.
- lmeyerov 6y agotyping pandas seems like an r&d-level problem as it seems quite close to tricky areas like row types / dependent types that mostly only work in theory attempting manual workarounds like pure nominal typing for it as you describe is against the grain of a type system and pushes a lot of work to the user. We just... don't. We stick with class ('pd DataFrame'), and the one extension we are thinking is Index, while getting into actual columns gets into a mess quite quickly. I tried to engage some PL researchers on this awhile back but no go. IMO would be a great project for most type system research students.
- hirple 6y agoNot exactly this, but I've had some success using logic programming (minikanren) to statically assert certain facts about dataframes in spark. E.g. "if Row_a is filtered to Row_b, and Row_a is not null, Row_b is also not null".