3 ms·
Ask HN: What do you use to data mung?
Hi! HN lover, tech enthusiast, and economist here.
I'm at a point in my skillset that I'd like to learn how to better data mung.
I can do basics in python, mysql, and with spreadsheets. I'd like to learn how to take things to the "next level", as it were. I seem to read about fantastic projects in data vis and collection at least daily here, and the projects are generally fantastic.
I find that collation of data seems to take forever at times; processes that should be simple require some convolutions for me. How can I data mung like a boss?
Thanks in advance for the heads ups!
- cynusx 13y agoseems to me like you need to learn some basic programming to cleanse and reformat data. Since you listed python here already, it may be that you already know some basic programming but if a program takes forever you probably need to learn about algorithmic complexity. You may program something that's in principal correct but may be very expensive to calculate. This link seems to cover the basics http://community.topcoder.com/tc?module=Static&d1=tutorials&d2=complexity1 http://community.topcoder.com/tc?module=Static&d1=tutorials&... So other than that, there's the R language that's pretty good for statistical analysis of data. If you're serious about data analysis, learning to program and either improving your skillset in python or learning a dedicated data analysis language like R will help the most.
- tomrod 13y agoThanks cynusx! I tend to write my own stats code with python, though I've toyed with R for a few papers. The integration via Rpy is excellent. I'll give your algorithms link a read. Thanks for the heads up!