3 ms·

It seems quite quick to do work on large datasets, I've found the documentation lacking however. There are no links in docs to the types that are being referen

It seems quite quick to do work on large datasets, I've found the documentation lacking however. There are no links in docs to the types that are being referenced, some types do not have documentation, some documentation is the function header with no other info ie no documentation, functions that take string formatting info eg '5min' do not have their argument possibilities documented anywhere I can find.

raymondh 12y ago
Really? You found the documentation to be lacking. FWIW, there are over 1500 pages in the docs including a short-over view, tutorials, extensive feature coverage, and interaction with other tools: http://pandas.pydata.org/pandas-docs/version/0.15.0/pandas.pdf http://pandas.pydata.org/pandas-docs/version/0.15.0/pandas.p... The docs may have some issues, but they certainly can't be characterized as lacking.
- Bootvis 12y ago
  In my eyes, you both make a valid point: yes there's plenty of documentation but sometimes I just can't find what I'm looking for. I like how the book[1] is structured, it really helped me but it isn't complete. Don't get me wrong, I'm grateful for all the work and I know I haven't contributed much but I think the online could be improved with more examples and recipes. Edit: There really is no excuse, getting started is easy[2]. [1]: http://shop.oreilly.com/product/0636920023784.do http://shop.oreilly.com/product/0636920023784.do [2]: http://pandas.pydata.org/developers.html http://pandas.pydata.org/developers.html
  - EmlynC 12y ago
    Correct me if I'm wrong but perhaps you had the same issue as me. The documentation is plentiful, lots of good examples and the book, similarly, increases with a nice linear complexity from basic "how do I select a (cell | row | column) ..." to full blown how do I do timeseries analysis on a dataseries pulled in from a remote source. The issue I had was not the documentation but the language of pandas mirrors the language used in R (I think this is something Wes McKinney intentional did) and it's the burden of all that new verbage that makes the documentation harder to sift through. Some choice exampels; "melt", "stack/unstack" and "reindex" — necessary, I grant you, so that functions can be aptly named and in turn encapsulate vectorised procedures that are composable. I found that the documentation was harder to search because I lacked the domain language and the documentation, for better for worse, doesn't dawdle with educating the reader about the verbage — worked examples often provide a easier route. It reads like a mathematical proof rather than prose and I used to think that the documentation was too terse but now I appreciate that probably just succinct.
RayVR 12y ago
Documentation is mostly organized around trying to explain how to use some specific feature, which is usually not the best format for me, but it may be for others. The argument possibilities has always been an issue for me. In general, I have found, if you have non-homogenous data, Pandas is your best bet due to how general it is, even if it is sometimes frustrating when it forces a generalization on your data (e.g., try doing type conversions on numpy.datetime64, it lacks any sort of intuition). The distinction between a Series and a dataframe is something I always found pretty silly/frustrating and I wonder if it was the result of an early implementation issue rather than a logical simplification. maybe I'm particularly ignorant of some issues since my use case is perhaps more straightforward than some others but I've built an entire labelled data library for my team and it is easier for us to operate under similar primitive beliefs to the numpy ndarray, i.e., it's always an ndarray, adding a column (or dimension) does not change the type of the object and the associated methods and indexing in one particular way vs another (df['a'] vs df[['a']]) does not change the type of your object. If I'm missing the point of Series I would love to see them justified or a use case referenced.

raymondh 12y ago

Really? You found the documentation to be lacking. FWIW, there are over 1500 pages in the docs including a short-over view, tutorials, extensive feature coverage, and interaction with other tools: http://pandas.pydata.org/pandas-docs/version/0.15.0/pandas.pdf http://pandas.pydata.org/pandas-docs/version/0.15.0/pandas.p... The docs may have some issues, but they certainly can't be characterized as lacking.

Bootvis 12y ago
In my eyes, you both make a valid point: yes there's plenty of documentation but sometimes I just can't find what I'm looking for. I like how the book[1] is structured, it really helped me but it isn't complete. Don't get me wrong, I'm grateful for all the work and I know I haven't contributed much but I think the online could be improved with more examples and recipes. Edit: There really is no excuse, getting started is easy[2]. [1]: http://shop.oreilly.com/product/0636920023784.do http://shop.oreilly.com/product/0636920023784.do [2]: http://pandas.pydata.org/developers.html http://pandas.pydata.org/developers.html
- EmlynC 12y ago
  Correct me if I'm wrong but perhaps you had the same issue as me. The documentation is plentiful, lots of good examples and the book, similarly, increases with a nice linear complexity from basic "how do I select a (cell | row | column) ..." to full blown how do I do timeseries analysis on a dataseries pulled in from a remote source. The issue I had was not the documentation but the language of pandas mirrors the language used in R (I think this is something Wes McKinney intentional did) and it's the burden of all that new verbage that makes the documentation harder to sift through. Some choice exampels; "melt", "stack/unstack" and "reindex" — necessary, I grant you, so that functions can be aptly named and in turn encapsulate vectorised procedures that are composable. I found that the documentation was harder to search because I lacked the domain language and the documentation, for better for worse, doesn't dawdle with educating the reader about the verbage — worked examples often provide a easier route. It reads like a mathematical proof rather than prose and I used to think that the documentation was too terse but now I appreciate that probably just succinct.

RayVR 12y ago

Documentation is mostly organized around trying to explain how to use some specific feature, which is usually not the best format for me, but it may be for others. The argument possibilities has always been an issue for me. In general, I have found, if you have non-homogenous data, Pandas is your best bet due to how general it is, even if it is sometimes frustrating when it forces a generalization on your data (e.g., try doing type conversions on numpy.datetime64, it lacks any sort of intuition). The distinction between a Series and a dataframe is something I always found pretty silly/frustrating and I wonder if it was the result of an early implementation issue rather than a logical simplification. maybe I'm particularly ignorant of some issues since my use case is perhaps more straightforward than some others but I've built an entire labelled data library for my team and it is easier for us to operate under similar primitive beliefs to the numpy ndarray, i.e., it's always an ndarray, adding a column (or dimension) does not change the type of the object and the associated methods and indexing in one particular way vs another (df['a'] vs df[['a']]) does not change the type of your object. If I'm missing the point of Series I would love to see them justified or a use case referenced.