2 ms·

Ask HN: Has anyone moved from data science to web dev? What was your experience?

6 points by jstx1 5y ago

PaulHoule 5y ago
I have gone back and forth. I'd say this. I've become convinced that the bottleneck in machine learning is the willingness of organizations to make training sets that are representative of their business problems. If you are downloading data sets from Kaggle you are going to be "always a bridesmaid and never a bride." So I am convinced that the royal road to automation is "break the task down into little bits, build something that people can control like a puppet to do the job, use the inputs to the puppet to train models that partially automate the task -- someday you need only 5-10% of the inputs that you started with" Thinking about it that way I am very happy to be mastering (and I do mean mastering) user interfaces with React and similar tools. I had a great opportunity in the past where I could have gotten management to take the initiative, formulate the problem correctly and build the right kind of training set but I didn't have the courage and I made the same mistake that almost everybody else makes to depend on generic algorithms and training sets -- something that we all knew would fail and it did. I won't be fooled again.
- jstx1 5y ago
  Did you switch within the same company or did you have to change companies? My concerns aren't so much about organisational problems but more about how to leverage existing experience when switching to a new problem domain.
  - PaulHoule 5y ago
    Changed companies, also worked for myself or as a consultant at times. Had some extended job hunts too. I am not doing any ‘data science’ at all in my current side projects which all revolve around art, images, manufacturing, etc. because that is my obsession now. But I guess I should take that back because I am building image selection pipelines again and these are building up training sets that one day will go into a model of some kind. (But that competes with colonizing my office walls, making digital twins for my supergraphics, making a persistence of vision display for my car, and devising a lightweight mirror I can install to perform with a pepper’s ghost, …) I think ‘data scientists’ themselves have more problems than the organizations they work in. Many of them think they are too good to have to think about reliable builds and the many details you have to attend to go from ‘I make a report’ to ‘I make a system that makes reports’. Some of them just won’t take orders when it comes to standardizing the way they do things. In web dev knowing how to use version control, issue tracking, etc. are table stakes. Discipline in software dev is important, just as important as knowing how to map click event coordinates to objects in a web UI, what to do about margin collapsing, where to keep state in a React app and not go insane,…
- f0e4c2f7 5y ago
  Hey I just wanted to say thanks for taking the time to write this comment. That's a really insightful view I hadn't considered but I think I agree with you and will likely spend more time with React and Vue now to have that card in my pocket for the future.
adingus 5y ago
I would love to learn more about this as well. I am working in data science and would like to switch to a backend web dev position. I work tangentially to back end devs and it seems like one of the biggest differences on the DS side is nobody knows anything about software development fundamentals. It's much harder to get good at these things when you need to learn them without knowledgeable team mates to challenge your ideas and not rubber stamp a PR.

6 points by jstx1 5y ago

I have gone back and forth. I'd say this. I've become convinced that the bottleneck in machine learning is the willingness of organizations to make training sets that are representative of their business problems. If you are downloading data sets from Kaggle you are going to be "always a bridesmaid and never a bride." So I am convinced that the royal road to automation is "break the task down into little bits, build something that people can control like a puppet to do the job, use the inputs to the puppet to train models that partially automate the task -- someday you need only 5-10% of the inputs that you started with" Thinking about it that way I am very happy to be mastering (and I do mean mastering) user interfaces with React and similar tools. I had a great opportunity in the past where I could have gotten management to take the initiative, formulate the problem correctly and build the right kind of training set but I didn't have the courage and I made the same mistake that almost everybody else makes to depend on generic algorithms and training sets -- something that we all knew would fail and it did. I won't be fooled again.

jstx1 5y ago
Did you switch within the same company or did you have to change companies? My concerns aren't so much about organisational problems but more about how to leverage existing experience when switching to a new problem domain.
- PaulHoule 5y ago
  Changed companies, also worked for myself or as a consultant at times. Had some extended job hunts too. I am not doing any ‘data science’ at all in my current side projects which all revolve around art, images, manufacturing, etc. because that is my obsession now. But I guess I should take that back because I am building image selection pipelines again and these are building up training sets that one day will go into a model of some kind. (But that competes with colonizing my office walls, making digital twins for my supergraphics, making a persistence of vision display for my car, and devising a lightweight mirror I can install to perform with a pepper’s ghost, …) I think ‘data scientists’ themselves have more problems than the organizations they work in. Many of them think they are too good to have to think about reliable builds and the many details you have to attend to go from ‘I make a report’ to ‘I make a system that makes reports’. Some of them just won’t take orders when it comes to standardizing the way they do things. In web dev knowing how to use version control, issue tracking, etc. are table stakes. Discipline in software dev is important, just as important as knowing how to map click event coordinates to objects in a web UI, what to do about margin collapsing, where to keep state in a React app and not go insane,…
f0e4c2f7 5y ago
Hey I just wanted to say thanks for taking the time to write this comment. That's a really insightful view I hadn't considered but I think I agree with you and will likely spend more time with React and Vue now to have that card in my pocket for the future.

adingus 5y ago

I would love to learn more about this as well. I am working in data science and would like to switch to a backend web dev position. I work tangentially to back end devs and it seems like one of the biggest differences on the DS side is nobody knows anything about software development fundamentals. It's much harder to get good at these things when you need to learn them without knowledgeable team mates to challenge your ideas and not rubber stamp a PR.