7 ms·

The talk title is quite provocative but the material discussed not so much. It is true that core data science is iterative, needs to reproducible and more recen

The talk title is quite provocative but the material discussed not so much. It is true that core data science is iterative, needs to reproducible and more recently explainable. Can you do everything in GUI? It really depends on where you see it from. Early data scientists were programmers before so they loved to code and built tooling around it. While code is still dominant we also see the rise of UI-centric tools - these allow you to build ML pipelines by snapping blocks together. I feel are chasing a "different" type of data scientist. The term data scientist itself has become quite broad. Code, CLI gives data scientists infinite flexibility but setup, management etc. is a challenge. GUI provides very less flexibility but you will have output fast - works for simpler problems IMHO. GUI or not data science has to move to cloud-based tools. Whether you write code in browser or CLI on local machine is matter of choice.

cup-of-tea 8y ago
> GUI or not data science has to move to cloud-based tools. This is a pretty odd statement. Care to explain why?
- raghavsb 8y ago
  Smaller datasets will work on laptop/desktop. For DL work with large datasets you need to build a GPU workstation. Moreover there setting up environment and dependencies on different hardware setups is not straight forward. Cloud provides the flexibility of choosing the hardware, many open source projects allow you to manage your dependencies and setup better. From 0 to something, cloud is better than custom.
  - PeterisP 8y ago
    For quite many companies their whole "big data" dataset is small enough to be processed by a single beefy machine. In that case it's far more cost effective to simply plug in $1000 worth of extra ram in a workstation rather than spend some extra engineer-hours to do it remotely.
    - raghavsb 8y ago
      Couldn't agree more on the data size. In most cases beefy machine work. Would on-demand (cloud) make it simple? Also beefy machine works for training jobs. But we need to deploy the models too.
      - cup-of-tea 8y ago
        With the many container solutions available today it's incredibly easy to move from dev to prod. You don't need to pay for a prod environment to do your development just to avoid ever having to migrate.
        anandology 8y ago
        Yes, it is incredibly easy, except when you upgrade to tensorflow 1.6 and it fails with [a cuda error][1] and after couple of sleepless nights you realize nvidia has deleted the docker image of cuda version 70xx from dockerhub and you need to find the right commit that works from their git repo and build everything yourself. [1]: https://github.com/tensorflow/tensorflow/issues/17566 https://github.com/tensorflow/tensorflow/issues/17566
  - walshemj 8y ago
    So if you struggle to "setting up environment and dependencies on different hardware setups" then maybe Data science and technical programming is not for you.
    - cup-of-tea 8y ago
      I wish this view was held more widely. Unfortunately the vast majority of "data science" jobs out there will not ask for any such technical ability.

cup-of-tea 8y ago

> GUI or not data science has to move to cloud-based tools. This is a pretty odd statement. Care to explain why?

raghavsb 8y ago
Smaller datasets will work on laptop/desktop. For DL work with large datasets you need to build a GPU workstation. Moreover there setting up environment and dependencies on different hardware setups is not straight forward. Cloud provides the flexibility of choosing the hardware, many open source projects allow you to manage your dependencies and setup better. From 0 to something, cloud is better than custom.
- PeterisP 8y ago
  For quite many companies their whole "big data" dataset is small enough to be processed by a single beefy machine. In that case it's far more cost effective to simply plug in $1000 worth of extra ram in a workstation rather than spend some extra engineer-hours to do it remotely.
  - raghavsb 8y ago
    Couldn't agree more on the data size. In most cases beefy machine work. Would on-demand (cloud) make it simple? Also beefy machine works for training jobs. But we need to deploy the models too.
    - cup-of-tea 8y ago
      With the many container solutions available today it's incredibly easy to move from dev to prod. You don't need to pay for a prod environment to do your development just to avoid ever having to migrate.
      - anandology 8y ago
        Yes, it is incredibly easy, except when you upgrade to tensorflow 1.6 and it fails with [a cuda error][1] and after couple of sleepless nights you realize nvidia has deleted the docker image of cuda version 70xx from dockerhub and you need to find the right commit that works from their git repo and build everything yourself. [1]: https://github.com/tensorflow/tensorflow/issues/17566 https://github.com/tensorflow/tensorflow/issues/17566
- walshemj 8y ago
  So if you struggle to "setting up environment and dependencies on different hardware setups" then maybe Data science and technical programming is not for you.
  - cup-of-tea 8y ago
    I wish this view was held more widely. Unfortunately the vast majority of "data science" jobs out there will not ask for any such technical ability.