wesm

15 ms·

121.
Pandas 0.4 (Python data analysis library) released(pandas.sourceforge.net)
84 points by wesm 15y ago|11 comments
122.
by wesm 15y ago
I've built the equivalent to most of the plyr and reshape/reshape2 packages inside pandas ( http://pandas.sourceforge.net , note I am in the midst of overhauling the documentation for the upcoming release). I plan to write a decent amount o
123.
by wesm 15y ago
We recently implemented the lowess smoother in statsmodels (pending pull request: https://github.com/statsmodels/statsmodels/pull/5 ), so it will work its way into the mix soon.
124.
SQL-like group by and aggregation using DataFrame in Python(wesmckinney.com)
5 points by wesm 15y ago|0 comments
125.
by wesm 15y ago
To be honest I haven't really broached the hierarchical columns issue inside pandas. If someone can look and suggest an implementation strategy that doesn't interfere with the rest of the API I would be all for it. If you have heterogeneous
126.
by wesm 15y ago
It is my project but I think pandas.DataFrame is already at 90+% feature-equivalency (especially if you're using the current git version...new release forthcoming). You don't have the integration with a million CRAN libraries. pandas.Data
127.
by wesm 15y ago
HDF5/PyTables is fantastic as a binary IO format. This is really all about in-memory data manipulation / computation and how data and metadata get passed around to functions that can take advantage of it.
128.
A Roadmap for Rich Scientific Data Structures in Python(wesmckinney.com)
70 points by wesm 15y ago|12 comments
129.
Podcast about enhanced data structures for statistical computing in Python(inscight.org)
2 points by wesm 15y ago|0 comments
130.
by wesm 16y ago
Picking on Python for not having simpler built-in ways to do domain-specific statistical operations seems rather silly to me. I've been involved the last few years with creating better data structures and tools for doing statistics in Pytho
131.
by wesm 16y ago
I agree-- and doing so could make some pretty serious waves across both academia and industry. You should join the discussion on the mailing lists: in particular pystatsmodels and the numpy/scipy lists. Any input would be much appreciated.
132.
by wesm 16y ago
Take a look at some of the really cool stuff at the last few SciPy conferences: http://conference.scipy.org/proceedings/SciPy2009/ http://conference.scipy.org/scipy2010/schedule.html also, consider joining the mailing lists I listed abo
133.
by wesm 16y ago
I'm actually working on a stats PhD (in the early stages) and I've made it my goal to build a lot of Python software as I go along. So if you have the inclination I'd recommend giving it a shot. When you consider the wealth of tools out the
134.
by wesm 16y ago
Depends on what you're referring to by "statistical process platforms". Python has been quite popular in the machine learning communities for several years, with some pure Python libraries (e.g. nltk, scikits.learn), and many wrappers to no
135.
by wesm 16y ago
R is a stable enough language to be used successfully in production systems. But I would definitely not recommend building anything "big" in R-- I've tried and the entire process was not pleasant. R is a research environment for statistic
136.
SciPy 2010: NumPy/SciPy coming to .NET(twitter.com)
2 points by wesm 16y ago|0 comments
137.
by wesm 17y ago
I used PyDev for a long time and recently switched to Wing IDE (had my company buy it for me). It's seriously like night and day when it comes to autocompletion and other IDE features that I found lacking in PyDev. Downside: not free.
138.
by wesm 18y ago
One of R's big benefits is the huge amount of statistical functionality, for example numerous different quantilization algorithms. If you are working with a lot of heterogeneous data in R it becomes a real headache. Merging data frames seem
139.
by wesm 18y ago
"One major advantage over Python is that it's vectorised, so you can say things like A + 1 when A is a vector (or matrix). A bit like Matlab in that regard, only that it doesn't suck as much as the Matlab ``language''." This is not very acc
140.
by wesm 18y ago
having programmed R on the job for some heavy statistics, I will say this: good for quick analyses but burdened by legacy functionality from the S+ days. I switched to Python/NumPy and rewrote all the R code I had, could not be happier with
141.
by wesm 18y ago
NumPy and other libraries aren't working 100% (have some breaking tests) on Python 2.6 yet due to some C API changes
142.
by wesm 18y ago
I think you'd be surprised at how powerful NumPy is. Try timing multiplying large matrices (I can tell you who wins, and not by a small margin) One downside is you do have to poke around for the right functions, whereas in Matlab you have
143.
by wesm 18y ago
I've had a lot of luck building scientific / research tools in Python / NumPy with Matplotlib. Integrating Fortran / C / C++ code is extremely easy as well, things like Cython / Pyrex also ease that quite a bit as well. My biggest complain
144.
by wesm 18y ago
Something huge in here in case you missed it: "To reduce memory usage, the garbage collector will now clear internal free lists when garbage-collecting the highest generation of objects. This may return memory to the operating system sooner