4 ms·

Astronomer to Data Scientist

39 points by gammarator 14y ago

elchief 14y ago
Some clarifications. 1. Java and Python are good languages for data mining, due to their libraries, though the others aren't great. C++ is good for finance. 2. Excel is not a statistical analysis package. "Just because you can drive a car with your feet doesn't make it a good idea" - Chris Rock. Excel is great for pivot tables, however. 3. Hadoop is not a distributed database, it is MapReduce plus a distributed file system. Hive is also not really a distributed database. It lets you write SQL that turns into mapreduce jobs.
- jwilliams 14y ago
  1. I wouldn't discount Perl. Depending on the industry it can be quite prevalent & it's very common as glue in others.
  - elchief 14y ago
    which Perl data mining libraries do you use/recommend?
    - jwilliams 14y ago
      The only one I've ever touched is BioPerl. It's well regarded in the Bioinformatics sphere.
- rjdagost 14y ago
  Excel is the most common statistical analysis package on the planet, bar-none. When you need to deliver code to people who are numerate but don't program then Excel is the best thing to use. Sometimes you just need to give someone (a manager) a tool where they can test out a bunch of different scenarios but you don't have time to make a polished application. Even for "big data" applications sometimes seeing the numbers in front of you in cells is quite useful.
  - elchief 14y ago
    Yes, it is popular, but that doesn't make it good for statistical analysis. It also sucks compared to JMP. On the accuracy of statistical procedures in Microsoft Excel 2007 http://or.nps.edu/faculty/PaulSanchez/oa4333/handouts/Excel/excel2007.pdf http://or.nps.edu/faculty/PaulSanchez/oa4333/handouts/Excel/...
- berkeleyjess 14y ago
  Thanks for the corrections. I have changed the post to reflect the Hadoop/Hive/MapReduce mistake. I am still new at this tech thing and obviously don't know all the terminology yet.
  - elchief 14y ago
    No worries, thanks for the post!
noelwelsh 14y ago
Looking from the other side, if I were hiring I wouldn't be too worried if the applicant knew this kind of stuff or not. It would be in their favour, but I'm far more interested in a strong mathematical background, intellectual horsepower, and personality. If they have these attributes (and any good researcher should) they should be able to learn the tech.
- berkeleyjess 14y ago
  I wish more people hiring had your attitude. I found that wasn't the case during my interview process.
evelyn1204 14y ago
http://www.magicdress.co.uk/3-formal-evening http://www.magicdress.co.uk/3-formal-evening
46Bit 14y ago
If you've hired/been hired for a "Data Scientist" role recently, what are the main skills and accomplishments you look for? I could do with a better idea as to whether we're talking someone who's a statistics fan or a distinguished PhD expert.
- deleted 14y ago
  [deleted]
- noelwelsh 14y ago
  At this point in time "Data Scientist" seems to be very loosely defined. I've seen everything from "must know Excel" to "must hold a PhD". You'll find lots of blog posts where people try to pin it down, but I don't think one definition will suffice. A more likely outcome is that we'll recognise specialisms within data science just like we have front-end, back-end, and other specialisms within programming.
skimmas 14y ago
when I looked at the site url the first thing I read was wo-meninas-astronomy, and meninas in portuguese is little girls. Ah :P funny

39 points by gammarator 14y ago

Some clarifications. 1. Java and Python are good languages for data mining, due to their libraries, though the others aren't great. C++ is good for finance. 2. Excel is not a statistical analysis package. "Just because you can drive a car with your feet doesn't make it a good idea" - Chris Rock. Excel is great for pivot tables, however. 3. Hadoop is not a distributed database, it is MapReduce plus a distributed file system. Hive is also not really a distributed database. It lets you write SQL that turns into mapreduce jobs.

jwilliams 14y ago
1. I wouldn't discount Perl. Depending on the industry it can be quite prevalent & it's very common as glue in others.
- elchief 14y ago
  which Perl data mining libraries do you use/recommend?
  - jwilliams 14y ago
    The only one I've ever touched is BioPerl. It's well regarded in the Bioinformatics sphere.
rjdagost 14y ago
Excel is the most common statistical analysis package on the planet, bar-none. When you need to deliver code to people who are numerate but don't program then Excel is the best thing to use. Sometimes you just need to give someone (a manager) a tool where they can test out a bunch of different scenarios but you don't have time to make a polished application. Even for "big data" applications sometimes seeing the numbers in front of you in cells is quite useful.
- elchief 14y ago
  Yes, it is popular, but that doesn't make it good for statistical analysis. It also sucks compared to JMP. On the accuracy of statistical procedures in Microsoft Excel 2007 http://or.nps.edu/faculty/PaulSanchez/oa4333/handouts/Excel/excel2007.pdf http://or.nps.edu/faculty/PaulSanchez/oa4333/handouts/Excel/...
berkeleyjess 14y ago
Thanks for the corrections. I have changed the post to reflect the Hadoop/Hive/MapReduce mistake. I am still new at this tech thing and obviously don't know all the terminology yet.
- elchief 14y ago
  No worries, thanks for the post!

noelwelsh 14y ago

Looking from the other side, if I were hiring I wouldn't be too worried if the applicant knew this kind of stuff or not. It would be in their favour, but I'm far more interested in a strong mathematical background, intellectual horsepower, and personality. If they have these attributes (and any good researcher should) they should be able to learn the tech.

berkeleyjess 14y ago
I wish more people hiring had your attitude. I found that wasn't the case during my interview process.

evelyn1204 14y ago

http://www.magicdress.co.uk/3-formal-evening http://www.magicdress.co.uk/3-formal-evening

46Bit 14y ago

If you've hired/been hired for a "Data Scientist" role recently, what are the main skills and accomplishments you look for? I could do with a better idea as to whether we're talking someone who's a statistics fan or a distinguished PhD expert.

deleted 14y ago
[deleted]
noelwelsh 14y ago
At this point in time "Data Scientist" seems to be very loosely defined. I've seen everything from "must know Excel" to "must hold a PhD". You'll find lots of blog posts where people try to pin it down, but I don't think one definition will suffice. A more likely outcome is that we'll recognise specialisms within data science just like we have front-end, back-end, and other specialisms within programming.

skimmas 14y ago

when I looked at the site url the first thing I read was wo-meninas-astronomy, and meninas in portuguese is little girls. Ah :P funny