5 ms·
Ask HN: best way to become a "Numerati"?
Not sure if "Numerati" is the best word. I already have a maths degree, but did little with statistics so far. I am capable of reading a book on statistics, but it is not the same as getting some practical experience.
I guess I could just start with randomly selected data sets, but somehow that doesn't seem to be enough motivation (like stock market data: seems unlikely to discover something new, with all the Quants already working on it). So I guess what I mean is, among other things: how to find worthwhile statistical problems for starting out? I am not into sports, so betting is kind of out. Also missed the elections, since I am not from the US. I look forward to the book by the 53something website guy, though.
- ntoshev 18y agohttp://netflixprize.com http://netflixprize.com Statistical language processing is also cool: http://nltk.sourceforge.net/index.php/Main_Page http://nltk.sourceforge.net/index.php/Main_Page http://norvig.com/spell-correct.html http://norvig.com/spell-correct.html
- hugh 18y agoIf numerati were a word, it'd be a plural. I'm reasonably sure the correct singular forms would be literatus (male) and literata (female). Oh, as for your actual question? I can't think of anything.
- scott_s 18y agoLook at graduate programs in statistics.
- Tichy 18y agoThat occured to me, too, but is it really necessary? Also, will a graduate program lead to practical applications?
- hugh 18y agoI'm sure it's not necessary. You haven't said why you want to learn statistics. Is it just for fun, or do you have some eventual practical application in mind? Personally, I just learn the bits I need, as I need them.
- scott_s 18y agoIt's never necessary, but it might be the easiest place to find interesting problems to work on, and the place that is most conducive to study. It's also probably the best lead-in to finding a job in the area. You don't have to do a PhD. Look up some Master's programs, which typically take two years. Disclosure: I'm a Computer Science PhD student.
- mechanical_fish 18y agoYou need to ask a biologist, a sociologist, or an epidemiologist. Or you could take up statistical process control. Useful skill, and applicable to many things besides semiconductor manufacturing. Here's a hilarious quote that I've remembered for years, from http://www.pbs.org/cringely/pulpit/2003/pulpit_20030925_000448.html http://www.pbs.org/cringely/pulpit/2003/pulpit_20030925_0004... "I taught over 300 courses for industry where we designed cars and electronic devices, but it wasn't until one day I took over my wife's kitchen and used Taguchi to perfect my recipe for vanilla wafer cookies that I realized how broadly it could be applied," Kowalick recalls. "It took 16 batches, but by the end of the afternoon I had those wafers dialed in." Go forth and bake some cookies.
- fbbwsa 18y agoi think there are still plenty of fun things to find in stock market data. most quants are looking at the same problem. i play with market data a lot and you'd be amazed how uneducated the general public seems to be. also, i second ntoshev's recommendations for netflix prize -ish applications. collaborative filtering/clustering seems to be an inadequate science at best as far as i can tell.
- jbert 18y agoRead some good sources: http://www.badscience.net/2008/11/you-are-80-less-likely-to-die-from-a-meteor-landing-on-your-head-if-you-wear-a-bicycle-helmet-all-day/ http://www.badscience.net/2008/11/you-are-80-less-likely-to-... http://www.google.co.uk/search?hl=en&q=the+tiger+that+isn't&btnG=Google+Search&meta= http://www.google.co.uk/search?hl=en&q=the+tiger+that+is... Perhaps take some public health scares and look up the papers behind them (if they exist). Try and work out if the scares are justified or not, based on the evidence. Perhaps do some meta-analyses of studies to see if you can determine evidence of publication bias: http://en.wikipedia.org/wiki/Publication_bias http://en.wikipedia.org/wiki/Publication_bias Find something that interests you and Do Science To It.
- sammyo 18y agoIf you're a hacker you might look for a niche in the R project and do some development.
- helveticaman 18y agoI would still use stocks, like you said, for starting out. I don't know if you're unlikely to find new things, but it'll probably be more interesting than sports.
- dronethebone 18y agoLook at some fun stuff like the Monty Hall problem, and Benford's law. These are delightful examples of how counterintuitive probabilities and statistics can be. If either of these things surprise you, you are in the great majority, and unlikely to perform practical statistical work without first doing a great deal of studying. Nothing to be ashamed of, not even Gauss was born knowing this stuff.
- Tichy 18y agoWell I am familiar with these. I just never got my hands on a real world data analysis problem.