13 ms·

What I learned from looking at every AI/ML tool I could find

459 points by amrrs 6y ago

peatmoss 6y ago
I love this kind of software meta-analysis. I have idly dreamed of setting up a site that essentially did literature review of existing solutions and approaches. I thought it would be a good contribution for a language with strong academic roots like Racket. I suspect a lot of good libraries remain unwritten in part because ecosystems tend to grow organically. Organic is good, but I’m positive that even basic meta-analysis like linked article would lead to better libraries in more languages, with less labor expended.
- kfk 6y ago
  Same thing for enterprise solutions. How does Redshift stack up against Snowflake? What about choosing a dashboard tool within the various ones that exist today? Etc.
kjw 6y ago
A great survey of the current landscape of ML tool innovation. This reminds me of the explosion of "big data" tech, which feels like it started exploding in 2010 and peaked maybe five years later. If ML follows a similar cycle, then in perhaps five years, most tools will have receded into obscurity but a few frameworks and approaches will become dominant. Big stakes and justifies the prevalence and investment in OSS. Comparison of "big data" and "machine learning" in Google Trends -- https://trends.google.com/trends/explore?date=all&geo=US&q=big%20data,machine%20learning https://trends.google.com/trends/explore?date=all&geo=US&q=b...
- etaioinshrdlu 6y ago
  I literally worked at a company that did a major project called "Big Data". It was in 2013/2014. It used Hadoop. It was pretty useless :)
- rurban 6y ago
  It will be called ML winter, not AI winter.
ww520 6y ago
This is a good overview of the landscape. From what I see, even if there's an AI bubble burst in funding, a lot of the ML techniques and algorithms can still be really useful for general computing with a smaller budget. Deep learning with neural net might be too costly to run for most projects. Shallow learning algorithms are a lot more cost effective to add to general computing. Algorithms like Support Vector Machine or Decision Tree learning work surprisingly well in targeted areas.
- visarga 6y ago
  They are OK if your problem is based on tabular data, but not for images and text. But the current paradigm is to reuse a pre-trained model, it's less more data intensive. In some tasks if you have a good backbone model you just need one or a few training examples.
  - IanCal 6y ago
    SVMs absolutely work on text, TF-IDF + SVM is a very classic (and pretty solid) approach for classification. It's easily explainable and has known classes of problems.
jlbnjmn 6y ago
What a great write up. The high level survey of the landscape was insightful, and directly comparing the challenges of ML/AI with their "solved" counterparts was great. Excellent read. Bookmarked.
reureu 6y ago
Awesome work - it must have been super interesting to dive into so many difference companies! The current state work that you describe is super fascinating. Be wary of survivorship bias when you're using today's landscape to draw historical conclusions. If you're only looking at companies/tools that exist today, taking their founding date, and using that as a sign of what existed circa 2012, then you're going to miss the plethora of companies/tools that failed prior to today. Similarly, much of the recent explosion may only be because they haven't existed long enough to have failed.
stephc_int13 6y ago
Aside from the irrational but probably inevitable over-hype, I find this whole AI/ML space quite fascinating and frustrating at the same time. Most of the successful algorithms are much less complex/sophisticated than you might believe from the outside, and I believe that's a very good sign because simplicity scales. But on the other hand, there is a huge amount of brute-forcing, you can't really do serious research in this field without a few dozen millions to spare, and I think that's always a limit on the innovation potential. Unfortunately writing good code/finding new algorithms is only a small part of the problem. I also think that many people don't completely understand how weak this model is to process real-world data when there no reliable way for the system to self-train on it.
- visarga 6y ago
  > I also think that many people don't completely understand how weak this model is to process real-world data when there no reliable way for the system to self-train on it. True, but probably any data can be used in self-training if you have captured enough of it. ML models can handle sequences, grids, trees, graphs, point clouds and sets as input formats. As for needing self-training in general, humans also need it even though we have such well attuned priors on the world based on billions of years of evolution. Even so, it takes years before a human can utter a phrase or accomplish a complex task. Basically today ML works by interpolation, if you have enough data points to cover your space you're good. But if you want to extrapolate outside, you're screwed. <rant>What we need is powerful simulators (physical or even neural based) to expose ML models to a variety of tasks inside the target domain. A simulator is like a dynamic dataset. A ML agent inside an environment (a simulator) is much more similar to humans than a model + dataset. Of course simulation is just as hard as AI.</>
- cambalache 6y ago
  I agree with you. I was in the Michael Jordan camp, as in: Machine Learning is just statistical inference rebranded and with better marketing. There are lots of cool applications for sure but there is no "magic sauce" beyond your usual models obtained by error minimization. But lately, especially seeing the progress with the Attention/Transformer models I have been slowly letting myself convinced that maybe that is all there is behind human intelligence: Lots of priors encoded by evolution in the brain, and general quasi-brute-force mechanism for inference, deduction and imagination. Of course this is probably a naive view, but I think the main point will remain: Intelligence can be created or simulated by giving an agent the capability to extrapolate in a way that "makes sense" in the environment the agent will live.
nl 6y ago
I do this stuff foe my day job, and everything here rings true. Every part of the "build and use ML in production" workflow is horrible (unless maybe you work at Google). Firstly, the Datascience workflow is NOT the same as software engineering. Things like version control tools don't work properly (in every part: git on Jupyter notebooks doesn't work without hacks, versioning data is horrible, versioning models is horrible). Deployment is horrible. Sagemaker (and equivalents) provide the very base level of functionality needed, but are so separated from the feature engineering side that everyone ends up doing vast amounts of work to get something useful. Frameworks are horrible. TensorFlow did the upgrade to TF2, so half the examples on the web don't work anymore. The TF data loading abstractions are great - if you work at Google but so complicated to get basic examples going. PyTorch has a horrible deployment story. Every other frame work are either experimental research things and take months to make progress (JAX) or are so far behind modern work they are useless (MXNet). But the thing is dealing with all these issues is worth it because ultimately it does actually work.
- timClicks 6y ago
  I think that notebooks have been a bit of a sideways step in a data science workflow. They're accessible, but they're tremendously brittle.
  - nl 6y ago
    Notebooks are so much better than anything else I've used for data science. I have a traditional SWEng background and came into data science never having used them. I'd never go back. I'm not saying that they are impossible to improve, but as a general approach they are exactly right. They are "brittle" when viewed as a software artefact. But that's not really what they are (or should be).
- jiggunjer 6y ago
  Any experience with ML in MATLAB?
  - nl 6y ago
evilelectron 6y ago
Great write up. Thanks for posting this. There’s rumor that due to a large number of people taking ML courses, there will be far more people with ML skills than ML jobs. I hope this is true. There are so many areas where ML skills could be useful. The sad part would be that some industries would be changed forever. The animation industry for example might not even exist the way it is today.
- visarga 6y ago
  Don't think it's the case - there will be more desired ML projects than people capable of implementing them. ML is like electricity 100 years ago or programming 40 years ago, we haven't applied it yet to most problems of society.
  - alephu5 6y ago
    The problem is it's not as useful as many people seem to think. I often hear my colleagues suggest ML for anything remotely complicated, even something like "measuring body fat percentage using electricity" that in reality only needs a physical equation. I've even heard people suggest it for web scraping which seems absolutely crazy to me.
    - ieatpies 6y ago
      I could actually see a use case for web scraping. If you're after particular pieces of content that aren't accessed in a structured way, on a site that rate limits you to the point of being restrictive, maybe using a bit of NLP could help you rank links to click.
      - hadeson 6y ago
        I tried using OCR to scrap Facebook profiles by simulate web browsing behavior. It helps a lot in avoid account blocking but still too slow to be practical.
        6y ago
timClicks 6y ago
When is "tooling" worth buying? Perhaps it's a reflection of the kind of person that I am, but I would always rather build my pipeline internally rather than outsource it. That pipeline will probably become part of the critical path.
- knodi123 6y ago
  > Perhaps it's a reflection of the kind of person that I am Probably. Lots of companies just want the shortest path from here to profitability. If the cost/benefit equations justify it, then eventually they can justify the outlay of building their own tooling to replace outsourced pieces a bit at a time. I'm thinking of things like AWS Lambda. Once, I built a slim version that was enough to support my own company's use case. I was told "that's good. save the git repo. one day it may be worth it to add the redundancy and scaling we already get from aws. until then, there's more profitable things for you to be working on."
  - anonytrary 6y ago
    > until then, there's more profitable things for you to be working on. Sorry, but this gave me a slight chuckle. I think I can empathize I guess, but found your wording hilariously blunt.
Proven 6y ago
Apparently it's no longer possible to summarize anything in post title... So be it, I'll never find out.
proverbialbunny 6y ago
>A large portion of AI investment is in self-driving cars, and as fully autonomous vehicles are still far from being a commodity, some hypothesize that investors will lose hope in AI altogether. Google has freezed hiring for ML researchers. Uber laid off the research half of their AI team. There’s rumor that due to a large number of people taking ML courses, there will be far more people with ML skills than ML jobs. Ouch >.< As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at. Thankfully I have a rare skill set. It's not found in classes, books, or boot camps. On the downside there are not many jobs to choose from as companies don't know what I do can be done. (Time series classification and prediction, usually sensor data.) Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad.
- huac 6y ago
  # of people who want to do "machine learning" > # of ML jobs > # of people good at ML
  - proverbialbunny 6y ago
    There should be a law that require all ML and DS classes, books, and bootcamps start by manually labeling data.
  - kqr 6y ago
    Yes, I was going to say. Taking courses in "ML" is a great way to get to write a hyped-up term on your resume. Actually understanding and being able to write code for statistical learning is something else entirely. Edit: On the other hand, I think Carmack observed that as long as you get the sign right, you'll get something out of it, even if it won't learn very quickly. So maybe speed of implementation and learning is how skilled people will differentiate themselves?
    - proverbialbunny 6y ago
awinter-py 6y ago
I'd consume this style of outline for almost any topic Regardless of my interest in it So useful
darepublic 6y ago
I don't think AI is overhyped, though certainly there are many bubble companies within it. I believe like the value and applications for cutting edge models is still being discovered, and I would only feel relief if less people piled on the bandwagon, out of a selfish impulse; more for me.
spicyramen 6y ago
A good read for beginners. I believe the literature about ML state seems to be a little dated with current times. People and companies are at the phase of already implementing ML in their companies, versus figuring out things: How? The ML field gives you options for every user and different levels of expertise. Advanced user: TF or Pytorch, preprocessing: Beam or Spark. Accelerators: GPUs. Tools such as Bigquery ML and AutoML have helped users to get into ML by generating MVP in an easy way. In would love to hear more from people in the field.
thundergolfer 6y ago
> If you have to choose between engineering and ML, choose engineering. It’s easier for great engineers to pick up ML knowledge, but it’s a lot harder for ML experts to become great engineers. If you become an engineer who builds great tools for ML, I’d forever be in your debt. Is this true, or at least the majority view? Maybe it is because the comparison is unbalanced, because "great engineer" is on both sides but the ML part is "pick up ML knowledge" and "ML experts". I'd choose engineering because the employment opportunities are so much more favourable, not because it's easy to 'pick up ML knowledge'. I'm in the beginning of many hours of mathematics and statistics study so that I can become competent at ML, and in comparison to say, becoming good at git and python and AWS, this feels harder.
- grenoire 6y ago
  I do think it's true, and likely the majority view too. The saying has many, many alternatives (for quantitative finance, it's "you can teach a math PhD finance but you can't teach a finance PhD math [at the same level that a math PhD is at]") in different studies. I think it just boils down to knowing the fundamentals or not, and arguably a person who only studied ML will lack a wide range of tools in their toolkit to be able to tackle tangential problems that an engineer could (whereas the engineer already has the tools to build up their ML toolkit). In my anecdotal experience with teaching, knowing how to learn is heavily correlated with how general (as opposed to focused) the field the person is in.
- cdavid 6y ago
  I agree 100 %, and I started as a PhD in ML 15 years ago. DL, thanks to its "lego aspect", is getting ML commoditized fast for most applications out there. It hollows the gap between applying more or less directly existing models and pure research. There is a glut of engineers who think ML engineering is playing with TF or pytorch all day, retraining models on well defined datasets. The reality: you want to use computer vision to do something cool server side ? Just using resnet/etc. and fine tuning on your data, maybe using as an embedded space, will get you 90 & there. What's hard ? Identifying opportunities, and especially data. I know it is cliché, but identifying what kind of problems you can solve w/ the data you have, and being able to convince the business side that what you do is useful, all of that is very hard. Same for the ability to solve 'big problems' while delivering regular deliverable so that you're not seen as a cost center. How do you link model improvements to business impact ? The system side of it is still immature as well. I say that to all my reports, directs or indirects: hone your SWE skills, make sure you understand how you bring value to your company. Unless you're in the top 1 %, you're not going to survive with ML skills alone.
tedk-42 6y ago
I did a ML/AI course in university in 2011 or 2012. After the course, I had a much greater appreciation for how clever and complex the decision making process in our brain is and how difficult it is to get a computer to even do basic decision making based on a set of inputs. There's human creativity as well which a machine cannot truely do. I'm always cautious and a bit pessimistic when I hear companies advertising AI skills or functionality built into their product. Most of them are probably driven by people willing to be the next facebook and hype, rather than real technology. That same part of me is rejoicing in seeing the industry pay for this hype train. I personally think the AI revolution will come from a hardware innovation (quantum computing or something like that) rather than building these large datasets to train models. We don't fully understand how our brains work so trying to replicate this with a computer program is a tough ask.
wickerman 6y ago
I'm a data engineer, stopped trying to "upgrade" into ML years ago when I realised my skills were a lot more in need in the longer term than any ML specialists.
jean-malo 6y ago
Interesting article! However as a software dev working in this field it is missing quite a few tools. The first that comes to mind is Dataiku which is already a pretty big startup (~$150M raised) so I'm surprised the author missed it.
hivacruz 6y ago
Thanks for the list. In my case, when I started doing ML/DL projects, I was looking for many many tools that could help me organize/train/deploy my model. I remembered looking at Kedro for days which looks great, or MetaFlow from Netflix. But at the end, I did my own project's structure with a enough Docker monolith that includes MLFlow as well. This is the only tool I use (with sklearn) for something else that training and serving my model. I think it's nice to do all of these by yourself at first and later use some of the tools listed there to enhance our workflow. It was really hard for me to get started with Kedro when your project is not as simple as MNIST or CIFAR.
TrackerFF 6y ago
The problem with investors is that they've been oversold on the "AI" part, and are expecting magic robots that will replace humans in a few years. Remember, the key motivation for investing in AI or ML is to become a more competitive business, by reducing (or eliminating) human labor costs. That's it. Luckily, ML has become very incorporated in the modern workforce, but mostly as part of specific tools that can eliminate easy and tedious tasks - just as intended. So yeah, it's sad that the big investor bucks have to come from more "exciting" projects, because there's a vast landscape of problems that can be aided my ML, even though they're not as exciting as self-driving cars, robot doctors, or whatever - and don't require some monstrous deep learning model to be solved.
guildmaster 6y ago
I've have been working on software engineering, web development primarily over the last few years. It's always felt that use of AI in industry was hyped up and soon the hype wind down. Finally there would be done very specific domains were AI can do exceptionally well. But recently I've been looking into the works of OpenAI and I can say that all my assumptions were out right wrong. The rate of growth in AI is increasing definitely. Just consider the voice tech landscape, with the new generative models around speech I can totally foresee the numbers of industries that are going to be disrupted from the ground up.
svaha1728 6y ago
`The pretrained large BERT model has 340M parameters and is 1.35GB. Even if it can fit on a consumer device (e.g. your phone), the time it takes for BERT to run inference on a new sample makes it useless for many real world applications. ` This is true, but a fine-tuned Distilbert model will give close to that accuracy with a ~260MB file size.
SomeoneFromCA 6y ago
The most important AI company nowadays is almighty Nvidia.
baxtr 6y ago
I am kinda irritated that this analysis was not done using ML...
deleted 6y ago
[deleted]

459 points by amrrs 6y ago

I love this kind of software meta-analysis. I have idly dreamed of setting up a site that essentially did literature review of existing solutions and approaches. I thought it would be a good contribution for a language with strong academic roots like Racket. I suspect a lot of good libraries remain unwritten in part because ecosystems tend to grow organically. Organic is good, but I’m positive that even basic meta-analysis like linked article would lead to better libraries in more languages, with less labor expended.

kfk 6y ago
Same thing for enterprise solutions. How does Redshift stack up against Snowflake? What about choosing a dashboard tool within the various ones that exist today? Etc.

kjw 6y ago

A great survey of the current landscape of ML tool innovation. This reminds me of the explosion of "big data" tech, which feels like it started exploding in 2010 and peaked maybe five years later. If ML follows a similar cycle, then in perhaps five years, most tools will have receded into obscurity but a few frameworks and approaches will become dominant. Big stakes and justifies the prevalence and investment in OSS. Comparison of "big data" and "machine learning" in Google Trends -- https://trends.google.com/trends/explore?date=all&geo=US&q=big%20data,machine%20learning https://trends.google.com/trends/explore?date=all&geo=US&q=b...

etaioinshrdlu 6y ago
I literally worked at a company that did a major project called "Big Data". It was in 2013/2014. It used Hadoop. It was pretty useless :)
rurban 6y ago
It will be called ML winter, not AI winter.

ww520 6y ago

This is a good overview of the landscape. From what I see, even if there's an AI bubble burst in funding, a lot of the ML techniques and algorithms can still be really useful for general computing with a smaller budget. Deep learning with neural net might be too costly to run for most projects. Shallow learning algorithms are a lot more cost effective to add to general computing. Algorithms like Support Vector Machine or Decision Tree learning work surprisingly well in targeted areas.

visarga 6y ago
They are OK if your problem is based on tabular data, but not for images and text. But the current paradigm is to reuse a pre-trained model, it's less more data intensive. In some tasks if you have a good backbone model you just need one or a few training examples.
- IanCal 6y ago
  SVMs absolutely work on text, TF-IDF + SVM is a very classic (and pretty solid) approach for classification. It's easily explainable and has known classes of problems.

jlbnjmn 6y ago

What a great write up. The high level survey of the landscape was insightful, and directly comparing the challenges of ML/AI with their "solved" counterparts was great. Excellent read. Bookmarked.

reureu 6y ago

Awesome work - it must have been super interesting to dive into so many difference companies! The current state work that you describe is super fascinating. Be wary of survivorship bias when you're using today's landscape to draw historical conclusions. If you're only looking at companies/tools that exist today, taking their founding date, and using that as a sign of what existed circa 2012, then you're going to miss the plethora of companies/tools that failed prior to today. Similarly, much of the recent explosion may only be because they haven't existed long enough to have failed.

stephc_int13 6y ago

Aside from the irrational but probably inevitable over-hype, I find this whole AI/ML space quite fascinating and frustrating at the same time. Most of the successful algorithms are much less complex/sophisticated than you might believe from the outside, and I believe that's a very good sign because simplicity scales. But on the other hand, there is a huge amount of brute-forcing, you can't really do serious research in this field without a few dozen millions to spare, and I think that's always a limit on the innovation potential. Unfortunately writing good code/finding new algorithms is only a small part of the problem. I also think that many people don't completely understand how weak this model is to process real-world data when there no reliable way for the system to self-train on it.

visarga 6y ago
> I also think that many people don't completely understand how weak this model is to process real-world data when there no reliable way for the system to self-train on it. True, but probably any data can be used in self-training if you have captured enough of it. ML models can handle sequences, grids, trees, graphs, point clouds and sets as input formats. As for needing self-training in general, humans also need it even though we have such well attuned priors on the world based on billions of years of evolution. Even so, it takes years before a human can utter a phrase or accomplish a complex task. Basically today ML works by interpolation, if you have enough data points to cover your space you're good. But if you want to extrapolate outside, you're screwed. <rant>What we need is powerful simulators (physical or even neural based) to expose ML models to a variety of tasks inside the target domain. A simulator is like a dynamic dataset. A ML agent inside an environment (a simulator) is much more similar to humans than a model + dataset. Of course simulation is just as hard as AI.</>
cambalache 6y ago
I agree with you. I was in the Michael Jordan camp, as in: Machine Learning is just statistical inference rebranded and with better marketing. There are lots of cool applications for sure but there is no "magic sauce" beyond your usual models obtained by error minimization. But lately, especially seeing the progress with the Attention/Transformer models I have been slowly letting myself convinced that maybe that is all there is behind human intelligence: Lots of priors encoded by evolution in the brain, and general quasi-brute-force mechanism for inference, deduction and imagination. Of course this is probably a naive view, but I think the main point will remain: Intelligence can be created or simulated by giving an agent the capability to extrapolate in a way that "makes sense" in the environment the agent will live.

nl 6y ago

I do this stuff foe my day job, and everything here rings true. Every part of the "build and use ML in production" workflow is horrible (unless maybe you work at Google). Firstly, the Datascience workflow is NOT the same as software engineering. Things like version control tools don't work properly (in every part: git on Jupyter notebooks doesn't work without hacks, versioning data is horrible, versioning models is horrible). Deployment is horrible. Sagemaker (and equivalents) provide the very base level of functionality needed, but are so separated from the feature engineering side that everyone ends up doing vast amounts of work to get something useful. Frameworks are horrible. TensorFlow did the upgrade to TF2, so half the examples on the web don't work anymore. The TF data loading abstractions are great - if you work at Google but so complicated to get basic examples going. PyTorch has a horrible deployment story. Every other frame work are either experimental research things and take months to make progress (JAX) or are so far behind modern work they are useless (MXNet). But the thing is dealing with all these issues is worth it because ultimately it does actually work.

timClicks 6y ago
I think that notebooks have been a bit of a sideways step in a data science workflow. They're accessible, but they're tremendously brittle.
- nl 6y ago
  Notebooks are so much better than anything else I've used for data science. I have a traditional SWEng background and came into data science never having used them. I'd never go back. I'm not saying that they are impossible to improve, but as a general approach they are exactly right. They are "brittle" when viewed as a software artefact. But that's not really what they are (or should be).
jiggunjer 6y ago
Any experience with ML in MATLAB?
- nl 6y ago

evilelectron 6y ago

Great write up. Thanks for posting this. There’s rumor that due to a large number of people taking ML courses, there will be far more people with ML skills than ML jobs. I hope this is true. There are so many areas where ML skills could be useful. The sad part would be that some industries would be changed forever. The animation industry for example might not even exist the way it is today.

visarga 6y ago
Don't think it's the case - there will be more desired ML projects than people capable of implementing them. ML is like electricity 100 years ago or programming 40 years ago, we haven't applied it yet to most problems of society.
- alephu5 6y ago
  The problem is it's not as useful as many people seem to think. I often hear my colleagues suggest ML for anything remotely complicated, even something like "measuring body fat percentage using electricity" that in reality only needs a physical equation. I've even heard people suggest it for web scraping which seems absolutely crazy to me.
  - ieatpies 6y ago
    I could actually see a use case for web scraping. If you're after particular pieces of content that aren't accessed in a structured way, on a site that rate limits you to the point of being restrictive, maybe using a bit of NLP could help you rank links to click.
    - hadeson 6y ago
      I tried using OCR to scrap Facebook profiles by simulate web browsing behavior. It helps a lot in avoid account blocking but still too slow to be practical.
      - 6y ago

timClicks 6y ago

When is "tooling" worth buying? Perhaps it's a reflection of the kind of person that I am, but I would always rather build my pipeline internally rather than outsource it. That pipeline will probably become part of the critical path.

knodi123 6y ago
> Perhaps it's a reflection of the kind of person that I am Probably. Lots of companies just want the shortest path from here to profitability. If the cost/benefit equations justify it, then eventually they can justify the outlay of building their own tooling to replace outsourced pieces a bit at a time. I'm thinking of things like AWS Lambda. Once, I built a slim version that was enough to support my own company's use case. I was told "that's good. save the git repo. one day it may be worth it to add the redundancy and scaling we already get from aws. until then, there's more profitable things for you to be working on."
- anonytrary 6y ago
  > until then, there's more profitable things for you to be working on. Sorry, but this gave me a slight chuckle. I think I can empathize I guess, but found your wording hilariously blunt.

Proven 6y ago

Apparently it's no longer possible to summarize anything in post title... So be it, I'll never find out.

proverbialbunny 6y ago

>A large portion of AI investment is in self-driving cars, and as fully autonomous vehicles are still far from being a commodity, some hypothesize that investors will lose hope in AI altogether. Google has freezed hiring for ML researchers. Uber laid off the research half of their AI team. There’s rumor that due to a large number of people taking ML courses, there will be far more people with ML skills than ML jobs. Ouch >.< As someone who does ML for a living this hits. When switching companies in 2019 for the first time I had a hard time despite having 10+ years experience. Linkedin maxed out saying over 200 people had applied to every job I was looking at. Thankfully I have a rare skill set. It's not found in classes, books, or boot camps. On the downside there are not many jobs to choose from as companies don't know what I do can be done. (Time series classification and prediction, usually sensor data.) Hopefully the industry matures. I've bumped into data scientists who don't know how to do feature engineering as well as ones who can't grok ML. It makes the whole industry look bad.

huac 6y ago
# of people who want to do "machine learning" > # of ML jobs > # of people good at ML
- proverbialbunny 6y ago
  There should be a law that require all ML and DS classes, books, and bootcamps start by manually labeling data.
- kqr 6y ago
  Yes, I was going to say. Taking courses in "ML" is a great way to get to write a hyped-up term on your resume. Actually understanding and being able to write code for statistical learning is something else entirely. Edit: On the other hand, I think Carmack observed that as long as you get the sign right, you'll get something out of it, even if it won't learn very quickly. So maybe speed of implementation and learning is how skilled people will differentiate themselves?
  - proverbialbunny 6y ago

awinter-py 6y ago

I'd consume this style of outline for almost any topic Regardless of my interest in it So useful

darepublic 6y ago

I don't think AI is overhyped, though certainly there are many bubble companies within it. I believe like the value and applications for cutting edge models is still being discovered, and I would only feel relief if less people piled on the bandwagon, out of a selfish impulse; more for me.

spicyramen 6y ago

A good read for beginners. I believe the literature about ML state seems to be a little dated with current times. People and companies are at the phase of already implementing ML in their companies, versus figuring out things: How? The ML field gives you options for every user and different levels of expertise. Advanced user: TF or Pytorch, preprocessing: Beam or Spark. Accelerators: GPUs. Tools such as Bigquery ML and AutoML have helped users to get into ML by generating MVP in an easy way. In would love to hear more from people in the field.

thundergolfer 6y ago

> If you have to choose between engineering and ML, choose engineering. It’s easier for great engineers to pick up ML knowledge, but it’s a lot harder for ML experts to become great engineers. If you become an engineer who builds great tools for ML, I’d forever be in your debt. Is this true, or at least the majority view? Maybe it is because the comparison is unbalanced, because "great engineer" is on both sides but the ML part is "pick up ML knowledge" and "ML experts". I'd choose engineering because the employment opportunities are so much more favourable, not because it's easy to 'pick up ML knowledge'. I'm in the beginning of many hours of mathematics and statistics study so that I can become competent at ML, and in comparison to say, becoming good at git and python and AWS, this feels harder.

grenoire 6y ago
I do think it's true, and likely the majority view too. The saying has many, many alternatives (for quantitative finance, it's "you can teach a math PhD finance but you can't teach a finance PhD math [at the same level that a math PhD is at]") in different studies. I think it just boils down to knowing the fundamentals or not, and arguably a person who only studied ML will lack a wide range of tools in their toolkit to be able to tackle tangential problems that an engineer could (whereas the engineer already has the tools to build up their ML toolkit). In my anecdotal experience with teaching, knowing how to learn is heavily correlated with how general (as opposed to focused) the field the person is in.
cdavid 6y ago
I agree 100 %, and I started as a PhD in ML 15 years ago. DL, thanks to its "lego aspect", is getting ML commoditized fast for most applications out there. It hollows the gap between applying more or less directly existing models and pure research. There is a glut of engineers who think ML engineering is playing with TF or pytorch all day, retraining models on well defined datasets. The reality: you want to use computer vision to do something cool server side ? Just using resnet/etc. and fine tuning on your data, maybe using as an embedded space, will get you 90 & there. What's hard ? Identifying opportunities, and especially data. I know it is cliché, but identifying what kind of problems you can solve w/ the data you have, and being able to convince the business side that what you do is useful, all of that is very hard. Same for the ability to solve 'big problems' while delivering regular deliverable so that you're not seen as a cost center. How do you link model improvements to business impact ? The system side of it is still immature as well. I say that to all my reports, directs or indirects: hone your SWE skills, make sure you understand how you bring value to your company. Unless you're in the top 1 %, you're not going to survive with ML skills alone.

tedk-42 6y ago

I did a ML/AI course in university in 2011 or 2012. After the course, I had a much greater appreciation for how clever and complex the decision making process in our brain is and how difficult it is to get a computer to even do basic decision making based on a set of inputs. There's human creativity as well which a machine cannot truely do. I'm always cautious and a bit pessimistic when I hear companies advertising AI skills or functionality built into their product. Most of them are probably driven by people willing to be the next facebook and hype, rather than real technology. That same part of me is rejoicing in seeing the industry pay for this hype train. I personally think the AI revolution will come from a hardware innovation (quantum computing or something like that) rather than building these large datasets to train models. We don't fully understand how our brains work so trying to replicate this with a computer program is a tough ask.

wickerman 6y ago

I'm a data engineer, stopped trying to "upgrade" into ML years ago when I realised my skills were a lot more in need in the longer term than any ML specialists.

jean-malo 6y ago

Interesting article! However as a software dev working in this field it is missing quite a few tools. The first that comes to mind is Dataiku which is already a pretty big startup (~$150M raised) so I'm surprised the author missed it.

hivacruz 6y ago

Thanks for the list. In my case, when I started doing ML/DL projects, I was looking for many many tools that could help me organize/train/deploy my model. I remembered looking at Kedro for days which looks great, or MetaFlow from Netflix. But at the end, I did my own project's structure with a enough Docker monolith that includes MLFlow as well. This is the only tool I use (with sklearn) for something else that training and serving my model. I think it's nice to do all of these by yourself at first and later use some of the tools listed there to enhance our workflow. It was really hard for me to get started with Kedro when your project is not as simple as MNIST or CIFAR.

TrackerFF 6y ago

The problem with investors is that they've been oversold on the "AI" part, and are expecting magic robots that will replace humans in a few years. Remember, the key motivation for investing in AI or ML is to become a more competitive business, by reducing (or eliminating) human labor costs. That's it. Luckily, ML has become very incorporated in the modern workforce, but mostly as part of specific tools that can eliminate easy and tedious tasks - just as intended. So yeah, it's sad that the big investor bucks have to come from more "exciting" projects, because there's a vast landscape of problems that can be aided my ML, even though they're not as exciting as self-driving cars, robot doctors, or whatever - and don't require some monstrous deep learning model to be solved.

guildmaster 6y ago

I've have been working on software engineering, web development primarily over the last few years. It's always felt that use of AI in industry was hyped up and soon the hype wind down. Finally there would be done very specific domains were AI can do exceptionally well. But recently I've been looking into the works of OpenAI and I can say that all my assumptions were out right wrong. The rate of growth in AI is increasing definitely. Just consider the voice tech landscape, with the new generative models around speech I can totally foresee the numbers of industries that are going to be disrupted from the ground up.

svaha1728 6y ago

`The pretrained large BERT model has 340M parameters and is 1.35GB. Even if it can fit on a consumer device (e.g. your phone), the time it takes for BERT to run inference on a new sample makes it useless for many real world applications. ` This is true, but a fine-tuned Distilbert model will give close to that accuracy with a ~260MB file size.

SomeoneFromCA 6y ago

The most important AI company nowadays is almighty Nvidia.

baxtr 6y ago

I am kinda irritated that this analysis was not done using ML...

deleted 6y ago

[deleted]