4 ms·

I would be interested to see more "low level" posts about how people approach their data analysis workflow. I do data analysis for a living and find that effici

by pav3l 14y ago

I would be interested to see more "low level" posts about how people approach their data analysis workflow. I do data analysis for a living and find that efficient workflow has just as much impact on your productivity and results as the choice in technologies that you use.

dude_abides 14y ago
That's a good idea; I will try and write up such a "low level" post this week. As a preview, the things that will prominently feature in the post are - RStudio, custom functions/aggregates in Postgres PL/R, and Sweave. (yes I am a R fanboy).
- pav3l 14y ago
  Would definitely like to read it! I personally find the need to go back and forth quite a bit between different tools when working even on a single dataset. Say, i'll do some work with text part of the dataset in Python, then use Matlab for some basic quantitative analysis, then use R for some advanced statistics if I have to, port those results back to Matlab/Python for speed, then back to R to sweave, maybe some spreadsheet stuff too, etc... So for me, for every project I need to have my dataset in easily accessible format (csv/someSQL/etc) and I need scripts in every language that I use, that would communicate with the data source and get "up to speed" right away. It's little tips like that I think people should be sharing more.
- washedup 14y ago
  RStudio for life. I would like to hear about your experience with Sweave.
Create 14y ago
http://www.r-bloggers.com/ http://www.r-bloggers.com/ make sure you look at some older posts
sadga 14y ago
Double your salary: call what you do "science", not "analysis".
- jmmcd 14y ago
  Maybe, but double it again by calling it "analytics".
  - mark_integerdsv 14y ago
    Can someone explain the affront to your sensibilities that is caused when one of our own (technologists) finds new ways to make money, rebrand and evolve a given subject area or just generally find new, better ways of explaining a thing? ...I'm baffled at the sarc and outrage every time something like this comes up. Data analyst is a massively broad term that could be anyone from a junior ad-hoc SQL script writer to a guy who sits on a DWH generating leads using complex models and algorithms. There is a need to distinguish these things from one another and the new definitions and tags are trying to suffice this need... what's your boggle?
    - jmmcd 14y ago
      No outrage, sarc, or boggle in my comment. New names are all good if they name new concepts. But new names for old things just sound like buzzwords.

by pav3l 14y ago

dude_abides 14y ago

That's a good idea; I will try and write up such a "low level" post this week. As a preview, the things that will prominently feature in the post are - RStudio, custom functions/aggregates in Postgres PL/R, and Sweave. (yes I am a R fanboy).

pav3l 14y ago
Would definitely like to read it! I personally find the need to go back and forth quite a bit between different tools when working even on a single dataset. Say, i'll do some work with text part of the dataset in Python, then use Matlab for some basic quantitative analysis, then use R for some advanced statistics if I have to, port those results back to Matlab/Python for speed, then back to R to sweave, maybe some spreadsheet stuff too, etc... So for me, for every project I need to have my dataset in easily accessible format (csv/someSQL/etc) and I need scripts in every language that I use, that would communicate with the data source and get "up to speed" right away. It's little tips like that I think people should be sharing more.
washedup 14y ago
RStudio for life. I would like to hear about your experience with Sweave.

Create 14y ago

http://www.r-bloggers.com/ http://www.r-bloggers.com/ make sure you look at some older posts

sadga 14y ago

Double your salary: call what you do "science", not "analysis".

jmmcd 14y ago
Maybe, but double it again by calling it "analytics".
- mark_integerdsv 14y ago
  Can someone explain the affront to your sensibilities that is caused when one of our own (technologists) finds new ways to make money, rebrand and evolve a given subject area or just generally find new, better ways of explaining a thing? ...I'm baffled at the sarc and outrage every time something like this comes up. Data analyst is a massively broad term that could be anyone from a junior ad-hoc SQL script writer to a guy who sits on a DWH generating leads using complex models and algorithms. There is a need to distinguish these things from one another and the new definitions and tags are trying to suffice this need... what's your boggle?
  - jmmcd 14y ago
    No outrage, sarc, or boggle in my comment. New names are all good if they name new concepts. But new names for old things just sound like buzzwords.