3 ms·

It's a really honest description of the attempts to move to neural networks. At least the way it's written it feels like no 'data scientists' were involved, it

It's a really honest description of the attempts to move to neural networks. At least the way it's written it feels like no 'data scientists' were involved, it was all done by data engineers (software developer rather than statistical/modelling knowledge)... Which is depressing, if even Airbnb are biased to hiring only good developers (rather than a mix)

PaulRobinson 7y ago
Most companies don't need to hire data scientists. They need to hire engineers who can take an algorithm implemented well by somebody else, and apply it to a business domain. That's the future of deep learning, machine learning, and all other linear/logistic regression style technologies. The mathematicians are going to have to wait for the age of quantum annealing to feel valuable again: reducing code into a function that works on a quantum setup actually needs those skills that developers struggle with. Everything else though, outside of pure research and in the vast majority of companies, is already well on the road to commoditisation.
- mlthoughts2018 7y ago
  This is pure comedy. Want to waste a bunch of cloud resources operationalizing a garbage model? Have engineers do it. The most successful workflow strategy I’ve seen in practice is where the deep learning researcher is also the person operationalizing the model. The same person who is grokking the latest paper in arxiv is also studying correlations in product data to perform feature engineering and also writing Dockerfiles to make the work reproducible and optimizing containers for production deployment, latency, failure tolerance, and evaluating performance in the specific context of the business application and creating well crafted software components with adequate testing along the way. The commodity part is the cloud engineering, kubernetes pod setup, load testing tools, and general software engineering. Machine learning engineers are typically great at these things and they are easy to learn. Meanwhile, learning about the nuance of hyperparameter tuning, how to evaluate overfitting, model complexity tradeoffs, when to use which kind of statistical modeling tool, how to improve models based on observing error cases, and a host of other statistical modeling concerns are wildly not commoditizable at this point of history. Knowing how to copy paste some Keras tutorials will not help you.
  - PaulRobinson 7y ago
    So I'll accept you might be right, except: 80% of real business problems are going to be solved by figuring out how to get XGBoost working with it. And you're done. Research is valuable, and there are some problems where doing real thinking is useful, but pareto principle is at play here: a lot of people just aren't going to need to do that, for the same reason most developers don't need to know the difference between a merge sort and a quick sort: sorting was commoditised into most programming languages decades ago. Same deal, different tech with machine learning. This is not a bad thing, and there will be a bumpy road, and there will always be a market for experts to help with the edge cases, but most firms will drop them like hot bricks within a decade.
- md8 7y ago
  "They need to hire engineers who can take an algorithm implemented well by somebody else, and apply it to a business domain." I dont think so. While Engineers are needed to operationalize the models. For Statistical Learning part, understanding of data preprocessing is an important step which requires knowledge in statistics.
baron_harkonnen 7y ago
I've got bad news for you, but the current state of rabid hiring means that this was very likely written by people that airbnb calls "data scientists" or "machine learning engineers", I'm sure half of these people have a phd in some arbitrarily "quantitative" field. The state of all large companies that I've seen that are not FAANG is that they are rushing to build teams of "data scientists" that slap together keras models and "ship" them, meaning the outputs are stuffed in a db only to be consumed by other keras models. My favorite gem from the original paper, which shows the sad, sad state of deep learning in industry is this line: >Out of sheer habit, we started our first model by initializing all weights and embeddings to zero, only to discover that is the worst way to start training a neural network. I can't imagine anyone who has even a mild understanding of how neural networks are implemented and trained making this mistake.

PaulRobinson 7y ago

Most companies don't need to hire data scientists. They need to hire engineers who can take an algorithm implemented well by somebody else, and apply it to a business domain. That's the future of deep learning, machine learning, and all other linear/logistic regression style technologies. The mathematicians are going to have to wait for the age of quantum annealing to feel valuable again: reducing code into a function that works on a quantum setup actually needs those skills that developers struggle with. Everything else though, outside of pure research and in the vast majority of companies, is already well on the road to commoditisation.

mlthoughts2018 7y ago
This is pure comedy. Want to waste a bunch of cloud resources operationalizing a garbage model? Have engineers do it. The most successful workflow strategy I’ve seen in practice is where the deep learning researcher is also the person operationalizing the model. The same person who is grokking the latest paper in arxiv is also studying correlations in product data to perform feature engineering and also writing Dockerfiles to make the work reproducible and optimizing containers for production deployment, latency, failure tolerance, and evaluating performance in the specific context of the business application and creating well crafted software components with adequate testing along the way. The commodity part is the cloud engineering, kubernetes pod setup, load testing tools, and general software engineering. Machine learning engineers are typically great at these things and they are easy to learn. Meanwhile, learning about the nuance of hyperparameter tuning, how to evaluate overfitting, model complexity tradeoffs, when to use which kind of statistical modeling tool, how to improve models based on observing error cases, and a host of other statistical modeling concerns are wildly not commoditizable at this point of history. Knowing how to copy paste some Keras tutorials will not help you.
- PaulRobinson 7y ago
  So I'll accept you might be right, except: 80% of real business problems are going to be solved by figuring out how to get XGBoost working with it. And you're done. Research is valuable, and there are some problems where doing real thinking is useful, but pareto principle is at play here: a lot of people just aren't going to need to do that, for the same reason most developers don't need to know the difference between a merge sort and a quick sort: sorting was commoditised into most programming languages decades ago. Same deal, different tech with machine learning. This is not a bad thing, and there will be a bumpy road, and there will always be a market for experts to help with the edge cases, but most firms will drop them like hot bricks within a decade.
md8 7y ago
"They need to hire engineers who can take an algorithm implemented well by somebody else, and apply it to a business domain." I dont think so. While Engineers are needed to operationalize the models. For Statistical Learning part, understanding of data preprocessing is an important step which requires knowledge in statistics.

baron_harkonnen 7y ago

I've got bad news for you, but the current state of rabid hiring means that this was very likely written by people that airbnb calls "data scientists" or "machine learning engineers", I'm sure half of these people have a phd in some arbitrarily "quantitative" field. The state of all large companies that I've seen that are not FAANG is that they are rushing to build teams of "data scientists" that slap together keras models and "ship" them, meaning the outputs are stuffed in a db only to be consumed by other keras models. My favorite gem from the original paper, which shows the sad, sad state of deep learning in industry is this line: >Out of sheer habit, we started our first model by initializing all weights and embeddings to zero, only to discover that is the worst way to start training a neural network. I can't imagine anyone who has even a mild understanding of how neural networks are implemented and trained making this mistake.