4 ms·

Hi, I'm an adjunct instructor at the data science bootcamp Zipfian Academy[1]. It seems there's still a lot of confusion as to what a data scientist is. Data

Hi, I'm an adjunct instructor at the data science bootcamp Zipfian Academy[1]. It seems there's still a lot of confusion as to what a data scientist is. Data Scientists are typically analysts who know some combination of matlab/python/R that come up with predictive models to achieve some sort of business objective. This is usually related to a businesses' profit center. The work is typically anything beyond A/B testing using basic classifiers to figure out things like churn prediction, handling data quality, all the way to doing object recognition. Data Engineers typically work on the JVM/distributed systems to handle data at scale and implement models for data scientists in production. With respect to deep learning, I'm also the author of a java based distributed deep learning solution called deeplearning4j[2] I think people throw around deep learning because it's the new thing, but it really can be more accurate. This is due to not needing to do feature engineering[3] and feature extraction[4] Also, if you're interested, I will be giving talks at both hadoop summit[5] and OSCon[6] this year around distributed deep learning if you have any specific questions as to what all of this stuff is. Data Science is an amazing field to be in right now. For those of you who are interested in neural networks in general, Ersatz holds a great meetup (videos/tech talks recorded!) Happy to answer questions! [1]: http://www.zipfianacademy.com/ http://www.zipfianacademy.com/ [2] http://deeplearning4j.org/ http://deeplearning4j.org/ [3]: http://www.cs.princeton.edu/courses/archive/spring10/cos424/slides/18-feat.pdf http://www.cs.princeton.edu/courses/archive/spring10/cos424/... [4]: http://en.wikipedia.org/wiki/Feature_extraction http://en.wikipedia.org/wiki/Feature_extraction [5]: http://hadoopsummit.org/san-jose/speakers/ http://hadoopsummit.org/san-jose/speakers/ [6]: http://www.oscon.com/oscon2014/public/schedule/detail/33709 http://www.oscon.com/oscon2014/public/schedule/detail/33709 [7]: http://www.meetup.com/SF-Neural-Network-Afficianados-Discussion-Group/ http://www.meetup.com/SF-Neural-Network-Afficianados-Discuss...

mailshanx 13y ago
w.r.t. deeplearning4j: Isn't it impractical to use multiple compute nodes for parallelization, rather than GPUs when it comes to training deep nets?
- agibsonccc 13y ago
  Distributed GPUs ;). I'll be adding GPU support here shortly. First I'm working on the core nets, since everything is vectorized, swapping out the matrix implementations will be easy. Mainly focusing on a point release first. Just need to add Recursive Neural Tensor Nets[1]. Afterwards, distributed GPUs will be cake with the support that both NVIDIA[2] and AMD[3] support.You will find distributed compute is what google does[4] Edit: I'd like to mention ersatz[5] here as well. Edit: Thanks for the correction David. [1]: http://nlp.stanford.edu/sentiment/ http://nlp.stanford.edu/sentiment/ [2]: http://www.jcuda.org/jcuda/jcublas/JCublas.html http://www.jcuda.org/jcuda/jcublas/JCublas.html [3]: http://developer.amd.com/tools-and-sdks/heterogeneous-computing/aparapi/ http://developer.amd.com/tools-and-sdks/heterogeneous-comput... [4]: http://research.google.com/archive/large_deep_networks_nips2012.html http://research.google.com/archive/large_deep_networks_nips2... [5]: http://ersatzlabs.com http://ersatzlabs.com
  - deleted 13y ago
    [deleted]
    - dave_sullivan 13y ago
      Since Ersatz got mentioned, I'd like to point out that we don't do true distributed GPU yet (it's planned). Most of the models support a single GPU with the RNN models supporting multiple GPUs on the same machine. Of course, that's still way way faster than your laptop. Distributing computation across GPUs has been addressed here: http://techblog.netflix.com/2014/02/distributed-neural-networks-with-gpus.html http://techblog.netflix.com/2014/02/distributed-neural-netwo... We looked at a similar scheme, but didn't see it as practical on something like AWS Also here: http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgCatanzaro_icml2013.pdf http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC... In this case, they are using infiniband, a high speed ethernet card/bus/whatever-you-call-it, in order to share data very quickly between multiple systems. This is basically like google's experiment but w/ GPUs (and thus way cheaper?) I'll let Adam speak to his strategy. It's an open research question but that doesn't mean there aren't any options or that those options are even terribly difficult given the right resources. Of course, real world performance is another story... whether you need and/or can use all that capacity, etc.
elliptic 13y ago
Seems like a somewhat artificial distinction. I'm responsible for coming up with my own models and implementing them in production.
datasci-fi 13y ago
How can you possibly teach data science in 12 weeks?
- agibsonccc 13y ago
  Little late on the reply. It's non stop 9-6pm everyday for 3 months. This would be you quitting your job to learn the whole data science stack (SQL all the way through machine learning) Since the focus is on practical/hands on, with only the needed theory, concepts are retained enough on a practical level to become a productive junior data scientist. Note that our acceptance rate is also very low though. We are bringing in people who already will have phds or some sort of a software engineering background. The job placement is worth it though. We are seeing avg salaries of 115-120k starting. The problems being solved are also really interesting.

mailshanx 13y ago

w.r.t. deeplearning4j: Isn't it impractical to use multiple compute nodes for parallelization, rather than GPUs when it comes to training deep nets?

agibsonccc 13y ago
Distributed GPUs ;). I'll be adding GPU support here shortly. First I'm working on the core nets, since everything is vectorized, swapping out the matrix implementations will be easy. Mainly focusing on a point release first. Just need to add Recursive Neural Tensor Nets[1]. Afterwards, distributed GPUs will be cake with the support that both NVIDIA[2] and AMD[3] support.You will find distributed compute is what google does[4] Edit: I'd like to mention ersatz[5] here as well. Edit: Thanks for the correction David. [1]: http://nlp.stanford.edu/sentiment/ http://nlp.stanford.edu/sentiment/ [2]: http://www.jcuda.org/jcuda/jcublas/JCublas.html http://www.jcuda.org/jcuda/jcublas/JCublas.html [3]: http://developer.amd.com/tools-and-sdks/heterogeneous-computing/aparapi/ http://developer.amd.com/tools-and-sdks/heterogeneous-comput... [4]: http://research.google.com/archive/large_deep_networks_nips2012.html http://research.google.com/archive/large_deep_networks_nips2... [5]: http://ersatzlabs.com http://ersatzlabs.com
- deleted 13y ago
  [deleted]
  - dave_sullivan 13y ago
    Since Ersatz got mentioned, I'd like to point out that we don't do true distributed GPU yet (it's planned). Most of the models support a single GPU with the RNN models supporting multiple GPUs on the same machine. Of course, that's still way way faster than your laptop. Distributing computation across GPUs has been addressed here: http://techblog.netflix.com/2014/02/distributed-neural-networks-with-gpus.html http://techblog.netflix.com/2014/02/distributed-neural-netwo... We looked at a similar scheme, but didn't see it as practical on something like AWS Also here: http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgCatanzaro_icml2013.pdf http://www.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgC... In this case, they are using infiniband, a high speed ethernet card/bus/whatever-you-call-it, in order to share data very quickly between multiple systems. This is basically like google's experiment but w/ GPUs (and thus way cheaper?) I'll let Adam speak to his strategy. It's an open research question but that doesn't mean there aren't any options or that those options are even terribly difficult given the right resources. Of course, real world performance is another story... whether you need and/or can use all that capacity, etc.

elliptic 13y ago

Seems like a somewhat artificial distinction. I'm responsible for coming up with my own models and implementing them in production.

datasci-fi 13y ago

How can you possibly teach data science in 12 weeks?

agibsonccc 13y ago
Little late on the reply. It's non stop 9-6pm everyday for 3 months. This would be you quitting your job to learn the whole data science stack (SQL all the way through machine learning) Since the focus is on practical/hands on, with only the needed theory, concepts are retained enough on a practical level to become a productive junior data scientist. Note that our acceptance rate is also very low though. We are bringing in people who already will have phds or some sort of a software engineering background. The job placement is worth it though. We are seeing avg salaries of 115-120k starting. The problems being solved are also really interesting.