4 ms·
The main problem (from my point of view) of python data science ecosystem is a complete lack of standardization on anything. You have ten different libraries t
by SirHumphrey 1y ago
The main problem (from my point of view) of python data science ecosystem is a complete lack of standardization on anything.
You have ten different libraries that try to behave like 4 other languages and the only point of standardization is that there is usually something like .to_numpy() function. This means that most of the time I was not solving any specific problem related to data processing, but just converting data from a format one library would understand to something another library would understand.
In Julia (a language with it's own problems, of course) things mostly just work. The library for calculating uncertainties interacts well with a library handling units and all this works fine with the piloting library, or libraries solving differential equations etc. In python, this required quite a lot of boilerplate.
- HdS84 1y agoR with its 4(?) class systems enters the chat.
- ChrisRackauckas 1y agoIt's at least 5 at this point.
- rienbdj 1y agoIn defense of R the class systems do have different characteristics and they are not deeply embedded in the language or anything.
- Evidlo 1y agoNobody has mentioned array-api (and data-apis more generally), which is trying to standardize the way people interact with arrays across the Python ecosystem. https://github.com/data-apis/array-api https://github.com/data-apis/array-api https://data-apis.org/blog/announcing_the_consortium/ https://data-apis.org/blog/announcing_the_consortium/
- bornfreddy 1y agoSounds like a great idea, but difficult to achieve. The announcement blog post was almost 5 years ago, do you know maybe what the impact of this project has been in practice?
- nickledave 1y agoThere are yearly releases of the standard https://data-apis.org/blog/ https://data-apis.org/blog/ and I often see it ref'd on issues in individual libraries (numpy, pytorch) See also funding from CZI (on the blog) Subjectively I do find it helps with consistency -- and when things are not consistent, it's easier to discover what's different and why edit: but I completely agree with this post and the follow-up from the same author on "dumbpy". At least at first blush, I need to read in more detail.