8 ms·

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I bel

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. You don't need a degree, but I think you do need to spend some time to get a deep enough understanding of what's going on under the hood, which often includes some math and takes time. This can be made accessible and there are plenty of good resources for that. But all these "become an AI pro by looking at some visualizations and copying this code" is maybe hurting more than it helps because it gives the illusion of understanding when it's actually not there. I wouldn't want people learning (solely) from this touching my production systems, writing blogs, or putting papers on arXiv.

throwgeorge 6y ago
>putting their result on arXiv so they can put AI research on their resume are you kidding? it's resulted in a huge amount of researchers publishing results in journals so they can apply to funding agencies that are clueless. I collaborate with a national lab (in the us) and the number of LDRD calls that I'm on where classically trained scientists propose completely clueless machine learning projects is very high. so i think you're being quite naive (or disingenuous) about who the real culprits are in inflating arxiv. also you can't throw up stuff on arxiv without a prof vouching for you.
s1t5 6y ago
Do you ever feel like all the noise influences the way you think about your own career? I work as a data scientist and sometimes find the hype so off-putting that I think that I should look for a role that's related to solving some optimization problems outside of ML, or a software engineering role in some completely different domain and work as a backend developer or something similar.
- gas9S9zw3P9c 6y ago
  Definitely. I came from a software engineering background into AI 6+ years ago, and I was lucky to be there at the right time. From purely a career perspective, leaving aside that this may be your life's passion, I believe that right now is the worst time to get started in AI. These is so much noise and competition but very few actual jobs or value created. It's just a research PR machine. Just as with Data Science, companies that think they need AI often just need better data collection, pipeline and infrastructure engineering. Good backend/infrastructure/data engineers are so much harder to find these days, and these skills IMO provide much more value than doing some kind of modeling.
  - MrBuddyCasino 6y ago
    > Good backend/infrastructure/data engineers are so much harder to find these days, and these skills IMO provide much more value than doing some kind of modeling. After all these years of FOMO on AI, this is music to my ears.
    - ricksharp 6y ago
      Right, I’ve been reading about machine learning for about 5 years, have read hundreds of articles about different techniques, and have often tried to explore ways it could be used. However, I’ve never found a practical use in software engineering. Every time I think I discover something that could use machine learning, I usually don’t have any data to work with or don’t have a clear definition of what the inputs and outputs would be. In the end, I find a way to develop a solution that doesn’t need AI and often makes me realize that AI would not have been able to provide the required reliability. I even participated in a self-driving RC car competition (running on raspi 3). I ended up scrapping the ML solution because it could only run at 10 hertz which was not even close to what was needed. I ended up writing a custom algorithm in C that ran at 200fps and could do “advanced” moves like backing up and course correcting. I’m not saying that ML doesn’t have its place. I’m saying that right now it’s all about images and nlp and has huge costs. In any practical usage, the ML parts that I would need are already available as a service that I can use almost free (like azure cognitive services).
chongli 6y ago
put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. That’s one way of looking at it. Perhaps another way is ask why all of these employers are hiring people who have done low quality research without understanding who they’re really hiring. It reminds me of the adverse market effects on gamers when Bitcoin miners were buying up all the GPUs a few years ago. It’s another emergent collective phenomenon that’s distorting the market. I think there may be quite a few employers out there with non-technical management who have become convinced that they need to hire a machine learning expert, without any particular reason why. They might hear a competitor has hired someone and so they need to as well. Really weird.
- gas9S9zw3P9c 6y ago
  I think you're absolutely right. In my experience 1. There is much more supply than demand in terms of Machine Learning and Data Science. There aren't actually that many jobs outside of research, it's just that the hype makes it look that way. Now that all these PhDs seem to be starting in ML, I wonder what they will do in 4-5 years when they finish the degree. I don't think a market for them will exist. 2. Many companies don't know what they are doing. They are hiring ML people because they want to put AI into their marketing materials. In reality, they don't need ML, they just need someone collecting data, building a database, and running a query. They just don't realize that's the case. The same happened with "Data Science" and "Big Data" - What most companies needed were software engineers building infrastructure and data collection, not people running sklearn.
  - bumby 6y ago
    > they want to put AI into their marketing materials. Somebody should start a thread that captures the silly examples of this. My personal favorite was a workout template “powered by AI”. Mind you, the only information the customer provided was the basics like age, sex, weight, and goal. This signaled peak AI hype for me.
  - deleted 6y ago
    [deleted]
- 6y ago
TrackerFF 6y ago
But then again, a lot of current ML jobs are basically just that - finding some optimal architecture, tune hyperparameters, and bam! You're now a modern "AI" powered company. Heck, I've encountered plenty of ML jobs that didn't require anything more than familiarity with some known frameworks or libraries, and being able to apply known methods to real-world data / problems. So I can absolutely understand why people are copy/pasting tutorials or papers, and just doing some slight changes. You're practically miles ahead of the competition, when it comes to the job search.
- mlthoughts2018 6y ago
  This is really a myth. Most ML jobs require very detailed understanding of statistics because the devil is in the details. You need to understand things like multicollinearity, coding biases, missing data techniques, convergence of Markov chains, learning curves, mechanics of various higher order gradient optimization methods, how to really carefully evaluate goodness of fit in a huge range of categories of models (neural nets are the vast minority of all models used in production settings) and a ton more beyond this. If you read “tensorflow for hackers” and believe you can write production neural nets, it’s a disaster. Garbage in equals garbage out.
  - TrackerFF 6y ago
    I can just tell from my own experience, having interviewed and researched a ton of ML jobs: The majority of ML jobs today seem to be re-branded analytics jobs. I'd say that a solid 4 in 5 of the jobs I've interviewed for, which were tagged in the ML domain, were just that. Typical [x] analytics jobs which don't really require more than stats 101, and good handling of excel. Basic scripting knowledge were often in the nice-to-know section. Now, there might be a world difference in the typical ML jobs you see in startup hubs like SF, and the jobs you see elsewhere - but companies are, and have been for almost 10 years, been desperate to get onboard of the hype-train, and have re-banded a lot of jobs to attract those wanting to work with ML or Data Science.
deleted 6y ago
[deleted]
mpfundstein 6y ago
by far thr worst part of Deep Learning research is the flood of crappy papers. i dont want to know how much time i wasted on reading through papers with math errors or trying to replicate results without success. and even if github code is available, i assure you. 95% of the time the code doesnt even work without major fixes. oh i hate it :-) so i started filtering. one pretty good filter is: only chinese authors from chinese universities? -> nope!
- dang 6y ago
  If you keep posting nationalistic flamebait to HN we are going to have to ban you. This has been a problem already: https://news.ycombinator.com/item?id=23363011 https://news.ycombinator.com/item?id=23363011.
melenaboija 6y ago
I agree in some sense but I also think that everybody has the right to learn at any level, mainly because not everything in neural networks field is research to create the next optimizer or a new architecture. This is perfect for the people I work with and the role we have (and I am talking about some of them are PhDs in math and physics but without much CS knowledge). Some of this people just need to see this is a stack of non linear functions that have to be minimized and they grasp the idea of nn really fast. To me this would be like saying that if you don't know computational complexity theory or how compilers work you are not a good software developer, but I think is just a different level and as long as you don't try to inflate your resume is fine. Sometimes I feel some fear on AI field when the knowledge is spread, and there is a bunch of backgrounds (maths, physics, quantitative finance, chemistry, ...) that need exactly this to at list demistify nn.
pachico 6y ago
I guess the same could be argued for coding, in general. I myself don't have a degree in CS yet I can tell amateur code from well designed one (unless I have fooled everyone in the last 20 years!). The industry has ways to filter sloppy work, somehow, I wouldn't worry too much about it. I would, instead, include the necessity of being able to judge good from bad in this fieald too, as it's always been for everything else.
amelius 6y ago
Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf https://pjreddie.com/media/files/papers/YOLOv3.pdf
- gas9S9zw3P9c 6y ago
  Oh yes, I don't think advanced math is required in any way. However, there is a difference in that researchers who have worked with these models for many years (often including having done some of the math) have a very good intuitive understanding of these models. Once you have that, it's fine to be driven by gut feeling. Just like many engineers are driven by gut feeling that come from tacit knowledge through experience. What is dangerous is reading a 30 minute blog post and getting the illusion of having some kind of understanding, when in reality it can take years to develop that. It's like cloning the postgres Github repository, compiling it, running a few queries, and then saying you've built databases and being hired to become the "database expert" at some company, spreading wrong knowledge left and right. That's why the popularization of these quick immediate reward tutorials is dangerous. It takes time and effort to become knowledgable at something. Of course, many people are smart enough to know that these tutorials (or cloning the postgres repo) is just the first step on a longer learning journey, and in that case it's totally fine, but there are also many people who start thinking they are experts ready to work on research or production models after going through such things, not being aware of the many things they don't know [0] [0] https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Dunning%E2%80%93Kruger_Effect_01.svg/1024px-Dunning%E2%80%93Kruger_Effect_01.svg.png https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Du...
  - shrimpx 6y ago
    It sounds like what you’re complaining about is some people who put papers on arxiv and deceivingly claim to be experts. And kind of implying that these blog posts are to blame, so the blog posts should be retracted due to those dishonest academic people? You’re making a very confusing point.
jfkebwjsbx 6y ago
The exact same thing happens in every non-regulated field. People want cheap everything, and a race to the bottom happens.
wolco 6y ago
Are you sure you want to start excluding and limiting this field? Part of the higher salaries is hype not necessarily value. That brings in the crowd who download change a parameter or two. They are not making world-wide breakthroughs but through that simple act they understand how things work better and can make more informed decisions around this topic. Any company who hires based on a simple hello world might be the company that doesn't even know why they want ml but follow the trend. This person might be the best candidate because they can think about ml on a slighly deeper level than the company and can help bridge that first step. They probably know the buzzwords, what's hot that's important to marketing.
sabalaba 6y ago
I would disagree that having a lot of interest hurts the field. Where are you seeing “noise and low quality work”? For your sake, I hope you’re not reading random papers from unknown authors on arXiv in your spare time! I think we’ve seen impressive contribution from people without PhDs. Chris Olah and Alec Radford come to mind first. (Note: I’m not implying that you disagree with that statement, just wanted to point some role models out to those without PhDs who want to contribute to the literature.) High quality work comes from a tiny incisive fraction of the research community. Most published research, even from PhDs, isn’t worth reading. Easily accessible tutorials promoting Colab are not the problem!
- currymj 6y ago
  there are many papers that seem like obvious crap that get into supposedly prestigious conferences (NeurIPS, ICML) every year. I think many people might reasonably believe that a paper in the supposed best venue in the field will be worth reading, but that is not really true. compared to machine learning, other research communities in computer science have much less nonsense. you can really feel the difference in quality if you look at papers accepted to equivalently competitive non-ML conferences. i think the broader problem may be that people hiring for very lucrative, competitive jobs are effectively outsourcing their hiring decisions to conference reviewers, which causes the whole system to break down.
  - p1esk 6y ago
    Link to at least one “obvious crap” paper that got accepted to NeurIPS or ICML?
    - wadkar 6y ago
      Shhh, he cannot reveal that he’s reviewer #2!
devalgo 6y ago
>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. I'm sorry but given that many papers in NeurIPS, ICML, etc are exactly what you described I find your criticism a bit lacking.
- gas9S9zw3P9c 6y ago
  The peer review system is so broken, especially with the number of submissions that the top journals get in ML now, that I don't even pay attention to what's accepted where anymore. It's all the same as arXiv to me. The best way to figure out what's useful and what's not is to wait and see which papers pass the test of time. But if you're an academic you don't necessarily have that luxury.
  - devalgo 6y ago
    Well that definitely speaks more to the field in general and not necessarily to "beginners" who tune a pretrained model and post the results to arXiv. There's massive incentives to post papers with incremental progress when there's potentially billions of dollars of Grants and VC money for folks with their names on those papers.
gumby 6y ago
I consider this equivalent to the democratization of web site development and then app development. It’s certainly lead to an explosion of crummy and security-nightmare apps but in exchange has been an on-ramp for some good developers and exciting products as well. The massive hype surrounding anything “AI” has caused the literature to become a dumpster fire, yes, but a handful of good papers still appear. Just use a low pass filter as with most things these days.
Datenstrom 6y ago
I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then: 1. Read "Artificial Intelligence A Modern Approach" and complete all the exercises [1] 2. Complete a ML course I recommend the Andrew Ng Coursera one [2] 3. Complete a DL course I recommend the Andrew Ng Coursera one [3] 4. Complete specialization courses in whatever you are interested in such as computer vision, reinforcement learning, natural language processing, etc. These will cover older traditional methods in the introduction also which will be very useful for determining when DL is not the correct solution. Additionally I suggest a data science course which usually covers other important things like data visualization and how to handle bad/missing data. Also, I learned a lot by simply being surrounded by brilliant people who know all this and being able to ask them questions and see how they approach problems. So not really self taught as much as untraditionally taught. Unfortunately not a single person has actually followed the advice. Everyone has only watched random youtube bloggers and read blogs. Some have gotten into trouble after landing a job by talking buzzwords and asked for help but my advice does not change. It does make it rather hard to find a job without a degree though, I would not recommend it. All of mine only come from strong references I have from luckily getting my foot in the door initially. [1]: http://aima.cs.berkeley.edu/index.html http://aima.cs.berkeley.edu/index.html [2]: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning [3]: https://www.coursera.org/specializations/deep-learning https://www.coursera.org/specializations/deep-learning Edit: formatting, typo
- gas9S9zw3P9c 6y ago
  Thanks for sharing your learning path and congrats to your success. There are so many great resources out there that it's possible for anyone to become an expert. Unfortunately, people like you who are willing to put in the hard work seem to be the minority. All of your success is well-deserved and props to you. Just like you said, most gravitate towards the easy-to-understand videos and blogs instead of confronting their gaps in knowledge. I've had the same experience with giving advice - anything that looks like it requires focused work (solving textbook problems!) or is unsexy is readily ignored in favor of the latest hype demo or high-level framework. > It was primarily driven by the need to solve problems for autonomous systems I was creating. I wonder if your success also had something to do with the fact that you had a specific problem you were trying to solve. Did you feel that your specific problem put everything you were learning into context and made it easier to stay motivated?
mumbisChungo 6y ago
>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. This seems extremely short-sighted to me. If the barrier to entry (on an already relatively young technology) has come down so far that there are a bunch of noobs running rampant right now, does that not bode extremely well for future advancements in the field (assuming some non-zero conversion rate from noobs to productive members of the field over time)? Also, let's not pretend that people with relevant degrees don't make shitty contributions to arXiv all the time.
- abnry 6y ago
  I'm surprised anyone would default to trusting arXiv papers, or even browse new additions, without first checking up on the credentials and experience of the authors. If the problem is finding new papers, then you really should be using sources other than arXiv.
sdinsn 6y ago
> At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. Note that this is not a problem exclusive to AI or Computer Science. I have a hobbyist interest in Entomology, and I was so disappointed to see that people are still pumping out papers that are minor tweaks off old population modeling papers from the 1980s. The field is shockingly stagnant. I've read random PhD theses from the 70s that are written on damn typewriters that are higher quality than modern papers from so-called "top tier" research universities.
AlexanderNull 6y ago
Yeah, if you're not programming your models in binary then gtfo imposters!! On a more serious note, yes understanding this and anything really takes time and investment. The problem for me at least originally not only with ML but originally with engineering back when I was trying to learn (and couldn't afford school), was finding quality sources for getting started in that process of learning. By providing simplified resources like this google one, the hope is that many beginners can get that one "aha!" moment where they start the basic understanding that allows them to start tinkering and learning. People without a decent understanding shouldn't be submitting research papers, full on agree there. It's basically a waste of everyone's time and harmful for the field of research as a whole as it dilutes the overall signal to noise ratio. However there's so much space in the ML world that doesn't involve research, not only for fun hobby projects, but also even professionally. Resources like this are critical to reducing the knowledge gap out there between researchers and the programmers in the field that work on little ML projects for doing things like sentiment analysis for their company. These sub-research projects are mission critical at a lot of companies yet are held up at the majority of non-FAANG companies because there's only one data scientist while the teams of engineers are clueless as to how to assist.
abhgh 6y ago
I share your sentiment to a fair extent and I have written about it before here on HN. Yes, for a lot of this stuff you don't need a PhD (and frankly I find that marketing weird for the above course). But you do need strong intuitions, understanding of some CS, and math "savviness" i.e. you don't have to know All The Math now, but you should be able to pick up stuff as needed, when you're trying to understand your problem and/or structuring your solution. One could learn all of this stuff online today - the amount of good resources out there is crazy. Frankly, I am jealous, because I began working on ML more than 10 yr ago, and we were relatively starved for resources on pretty much all fronts: resources to study from (reading material or videos), affordable compute power, s/w libraries. But unfortunately, despite their abundance today, most people don't take the time to dive deep. Of all the years of me suggesting courses and books to people (when asked), only ONE (or maybe two) person managed to go through them to a fair extent. But there is a significant fraction of the rest, for which reductive messaging like "become a pro in AI in 3 weeks" has been misleading. As a hiring manager sometimes they are as surprised as me, when an interview doesn't go well, after the resume seemed promising to both sides. And to be very clear, I don't blame them (sure, there are some pretentious opportunists, who flat out lie, but I've found them not to be the norm); all this messaging seems to have created a bubble where often you don't know what you don't know. It's amusing that thrice, rejected candidates reached out me saying that the interview was quite eye opening! I have been at the receiving end too - where 90% of the interview seemed to be about some very specific setting of a library or a method, because that was the conception of ML the interviewer had. I think people should learn, by whatever means, and create stuff because they can - this is the best kind of learning. Silly projects are great too, if they are fun - if they don't advance your understanding, they might motivate you to be less silly! What's missing in this ecosystem is honest messaging about where your skill levels really are. I don't know how to fix it in a way that also doesn't harm, in some way, the widespread learning/awareness reg ML. On a smaller scale though, I have accepted that this has increased my scope of work in screening resumes: if someone lists her github repo, or an arxiv paper, I actually need to spend time to go through them. I don't see this as noise, but a widening of the spectrum of available ML skills in the market; and I need to put in some effort to place an applicant in this spectrum. I've accepted that this is the flipside of working in a hot area: for the multiple job opportunities accessible to me, I have to put in more thought for hiring. I can't have the luxury of the former without the responsibility of the latter. Although, being lazy, I'd totally want to ;).
citilife 6y ago
I am one of those without a PhD, but have taken the time to the learn the math & contribute quite a bit to this area. That being said, I also don't think deep learning itself is not really a "science". The issue I have is you can't predict if a network will learn. We're effectively testing deep learning networks the same way the Romans used to test bridges. Send a bunch of elephants over them, if it holds it's good enough. There's obviously some indicators of success, but on a whole the overall interaction between components is very difficult to calculate and near-impossible to predict. While I think it's important to understand how layers interact and how a given function will impact your optimization, etc. it's not fully required to have a deep understanding of the mathematics, at least for most cases. I also personally don't view anything on arXiv worth anything. I typically will read articles/papers myself if reviewing a candidate and / or would like to see their publications at conferences or journals. Otherwise, it's essentially a blog post (which IMO is fine, but will require me to review it).
- andrewprock 6y ago
  The mathematics of neural networks is very well understood from a theoretical and scientific perspective. It is easy to say that a neural network will have predictive power given labelled training data. Whether or not the predictions it is making satisfy unstated requirements is another problem altogether.
  - deleted 6y ago
    [deleted]
- dumb1224 6y ago
  > We're effectively testing deep learning networks the same way the Romans used to test bridges. Send a bunch of elephants over them, if it holds it's good enough. Well for human level tasks maybe, but what about other areas of research where we like to discover patterns not seen by a human? Like finding links between genomic interplay with external perturbation such as radiotherapy? It'll be very lucky to have such 'elephants' to test at all.
  - 6y ago
zerr 6y ago
Maybe stopping using "AI" in favor of "Statistics" would help?
thallavajhula 6y ago
Isn't that the case with any aspect of Computer Science in general?
deleted 6y ago
[deleted]
Razengan 6y ago
> the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. Then the problem isn't the low barrier and ease of use, it's with arXiv's filtering and how resumes are valued and verified.

throwgeorge 6y ago

>putting their result on arXiv so they can put AI research on their resume are you kidding? it's resulted in a huge amount of researchers publishing results in journals so they can apply to funding agencies that are clueless. I collaborate with a national lab (in the us) and the number of LDRD calls that I'm on where classically trained scientists propose completely clueless machine learning projects is very high. so i think you're being quite naive (or disingenuous) about who the real culprits are in inflating arxiv. also you can't throw up stuff on arxiv without a prof vouching for you.

s1t5 6y ago

Do you ever feel like all the noise influences the way you think about your own career? I work as a data scientist and sometimes find the hype so off-putting that I think that I should look for a role that's related to solving some optimization problems outside of ML, or a software engineering role in some completely different domain and work as a backend developer or something similar.

gas9S9zw3P9c 6y ago
Definitely. I came from a software engineering background into AI 6+ years ago, and I was lucky to be there at the right time. From purely a career perspective, leaving aside that this may be your life's passion, I believe that right now is the worst time to get started in AI. These is so much noise and competition but very few actual jobs or value created. It's just a research PR machine. Just as with Data Science, companies that think they need AI often just need better data collection, pipeline and infrastructure engineering. Good backend/infrastructure/data engineers are so much harder to find these days, and these skills IMO provide much more value than doing some kind of modeling.
- MrBuddyCasino 6y ago
  > Good backend/infrastructure/data engineers are so much harder to find these days, and these skills IMO provide much more value than doing some kind of modeling. After all these years of FOMO on AI, this is music to my ears.
  - ricksharp 6y ago
    Right, I’ve been reading about machine learning for about 5 years, have read hundreds of articles about different techniques, and have often tried to explore ways it could be used. However, I’ve never found a practical use in software engineering. Every time I think I discover something that could use machine learning, I usually don’t have any data to work with or don’t have a clear definition of what the inputs and outputs would be. In the end, I find a way to develop a solution that doesn’t need AI and often makes me realize that AI would not have been able to provide the required reliability. I even participated in a self-driving RC car competition (running on raspi 3). I ended up scrapping the ML solution because it could only run at 10 hertz which was not even close to what was needed. I ended up writing a custom algorithm in C that ran at 200fps and could do “advanced” moves like backing up and course correcting. I’m not saying that ML doesn’t have its place. I’m saying that right now it’s all about images and nlp and has huge costs. In any practical usage, the ML parts that I would need are already available as a service that I can use almost free (like azure cognitive services).

chongli 6y ago

put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. That’s one way of looking at it. Perhaps another way is ask why all of these employers are hiring people who have done low quality research without understanding who they’re really hiring. It reminds me of the adverse market effects on gamers when Bitcoin miners were buying up all the GPUs a few years ago. It’s another emergent collective phenomenon that’s distorting the market. I think there may be quite a few employers out there with non-technical management who have become convinced that they need to hire a machine learning expert, without any particular reason why. They might hear a competitor has hired someone and so they need to as well. Really weird.

gas9S9zw3P9c 6y ago
I think you're absolutely right. In my experience 1. There is much more supply than demand in terms of Machine Learning and Data Science. There aren't actually that many jobs outside of research, it's just that the hype makes it look that way. Now that all these PhDs seem to be starting in ML, I wonder what they will do in 4-5 years when they finish the degree. I don't think a market for them will exist. 2. Many companies don't know what they are doing. They are hiring ML people because they want to put AI into their marketing materials. In reality, they don't need ML, they just need someone collecting data, building a database, and running a query. They just don't realize that's the case. The same happened with "Data Science" and "Big Data" - What most companies needed were software engineers building infrastructure and data collection, not people running sklearn.
- bumby 6y ago
  > they want to put AI into their marketing materials. Somebody should start a thread that captures the silly examples of this. My personal favorite was a workout template “powered by AI”. Mind you, the only information the customer provided was the basics like age, sex, weight, and goal. This signaled peak AI hype for me.
- deleted 6y ago
  [deleted]
6y ago

TrackerFF 6y ago

But then again, a lot of current ML jobs are basically just that - finding some optimal architecture, tune hyperparameters, and bam! You're now a modern "AI" powered company. Heck, I've encountered plenty of ML jobs that didn't require anything more than familiarity with some known frameworks or libraries, and being able to apply known methods to real-world data / problems. So I can absolutely understand why people are copy/pasting tutorials or papers, and just doing some slight changes. You're practically miles ahead of the competition, when it comes to the job search.

mlthoughts2018 6y ago
This is really a myth. Most ML jobs require very detailed understanding of statistics because the devil is in the details. You need to understand things like multicollinearity, coding biases, missing data techniques, convergence of Markov chains, learning curves, mechanics of various higher order gradient optimization methods, how to really carefully evaluate goodness of fit in a huge range of categories of models (neural nets are the vast minority of all models used in production settings) and a ton more beyond this. If you read “tensorflow for hackers” and believe you can write production neural nets, it’s a disaster. Garbage in equals garbage out.
- TrackerFF 6y ago
  I can just tell from my own experience, having interviewed and researched a ton of ML jobs: The majority of ML jobs today seem to be re-branded analytics jobs. I'd say that a solid 4 in 5 of the jobs I've interviewed for, which were tagged in the ML domain, were just that. Typical [x] analytics jobs which don't really require more than stats 101, and good handling of excel. Basic scripting knowledge were often in the nice-to-know section. Now, there might be a world difference in the typical ML jobs you see in startup hubs like SF, and the jobs you see elsewhere - but companies are, and have been for almost 10 years, been desperate to get onboard of the hype-train, and have re-banded a lot of jobs to attract those wanting to work with ML or Data Science.

deleted 6y ago

[deleted]

mpfundstein 6y ago

by far thr worst part of Deep Learning research is the flood of crappy papers. i dont want to know how much time i wasted on reading through papers with math errors or trying to replicate results without success. and even if github code is available, i assure you. 95% of the time the code doesnt even work without major fixes. oh i hate it :-) so i started filtering. one pretty good filter is: only chinese authors from chinese universities? -> nope!

dang 6y ago
If you keep posting nationalistic flamebait to HN we are going to have to ban you. This has been a problem already: https://news.ycombinator.com/item?id=23363011 https://news.ycombinator.com/item?id=23363011.

melenaboija 6y ago

I agree in some sense but I also think that everybody has the right to learn at any level, mainly because not everything in neural networks field is research to create the next optimizer or a new architecture. This is perfect for the people I work with and the role we have (and I am talking about some of them are PhDs in math and physics but without much CS knowledge). Some of this people just need to see this is a stack of non linear functions that have to be minimized and they grasp the idea of nn really fast. To me this would be like saying that if you don't know computational complexity theory or how compilers work you are not a good software developer, but I think is just a different level and as long as you don't try to inflate your resume is fine. Sometimes I feel some fear on AI field when the knowledge is spread, and there is a bunch of backgrounds (maths, physics, quantitative finance, chemistry, ...) that need exactly this to at list demistify nn.

pachico 6y ago

I guess the same could be argued for coding, in general. I myself don't have a degree in CS yet I can tell amateur code from well designed one (unless I have fooled everyone in the last 20 years!). The industry has ways to filter sloppy work, somehow, I wouldn't worry too much about it. I would, instead, include the necessity of being able to judge good from bad in this fieald too, as it's always been for everything else.

amelius 6y ago

Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf https://pjreddie.com/media/files/papers/YOLOv3.pdf

gas9S9zw3P9c 6y ago
Oh yes, I don't think advanced math is required in any way. However, there is a difference in that researchers who have worked with these models for many years (often including having done some of the math) have a very good intuitive understanding of these models. Once you have that, it's fine to be driven by gut feeling. Just like many engineers are driven by gut feeling that come from tacit knowledge through experience. What is dangerous is reading a 30 minute blog post and getting the illusion of having some kind of understanding, when in reality it can take years to develop that. It's like cloning the postgres Github repository, compiling it, running a few queries, and then saying you've built databases and being hired to become the "database expert" at some company, spreading wrong knowledge left and right. That's why the popularization of these quick immediate reward tutorials is dangerous. It takes time and effort to become knowledgable at something. Of course, many people are smart enough to know that these tutorials (or cloning the postgres repo) is just the first step on a longer learning journey, and in that case it's totally fine, but there are also many people who start thinking they are experts ready to work on research or production models after going through such things, not being aware of the many things they don't know [0] [0] https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Dunning%E2%80%93Kruger_Effect_01.svg/1024px-Dunning%E2%80%93Kruger_Effect_01.svg.png https://upload.wikimedia.org/wikipedia/commons/thumb/4/46/Du...
- shrimpx 6y ago
  It sounds like what you’re complaining about is some people who put papers on arxiv and deceivingly claim to be experts. And kind of implying that these blog posts are to blame, so the blog posts should be retracted due to those dishonest academic people? You’re making a very confusing point.

jfkebwjsbx 6y ago

The exact same thing happens in every non-regulated field. People want cheap everything, and a race to the bottom happens.

wolco 6y ago

Are you sure you want to start excluding and limiting this field? Part of the higher salaries is hype not necessarily value. That brings in the crowd who download change a parameter or two. They are not making world-wide breakthroughs but through that simple act they understand how things work better and can make more informed decisions around this topic. Any company who hires based on a simple hello world might be the company that doesn't even know why they want ml but follow the trend. This person might be the best candidate because they can think about ml on a slighly deeper level than the company and can help bridge that first step. They probably know the buzzwords, what's hot that's important to marketing.

sabalaba 6y ago

I would disagree that having a lot of interest hurts the field. Where are you seeing “noise and low quality work”? For your sake, I hope you’re not reading random papers from unknown authors on arXiv in your spare time! I think we’ve seen impressive contribution from people without PhDs. Chris Olah and Alec Radford come to mind first. (Note: I’m not implying that you disagree with that statement, just wanted to point some role models out to those without PhDs who want to contribute to the literature.) High quality work comes from a tiny incisive fraction of the research community. Most published research, even from PhDs, isn’t worth reading. Easily accessible tutorials promoting Colab are not the problem!

currymj 6y ago
there are many papers that seem like obvious crap that get into supposedly prestigious conferences (NeurIPS, ICML) every year. I think many people might reasonably believe that a paper in the supposed best venue in the field will be worth reading, but that is not really true. compared to machine learning, other research communities in computer science have much less nonsense. you can really feel the difference in quality if you look at papers accepted to equivalently competitive non-ML conferences. i think the broader problem may be that people hiring for very lucrative, competitive jobs are effectively outsourcing their hiring decisions to conference reviewers, which causes the whole system to break down.
- p1esk 6y ago
  Link to at least one “obvious crap” paper that got accepted to NeurIPS or ICML?
  - wadkar 6y ago
    Shhh, he cannot reveal that he’s reviewer #2!

devalgo 6y ago

gas9S9zw3P9c 6y ago
The peer review system is so broken, especially with the number of submissions that the top journals get in ML now, that I don't even pay attention to what's accepted where anymore. It's all the same as arXiv to me. The best way to figure out what's useful and what's not is to wait and see which papers pass the test of time. But if you're an academic you don't necessarily have that luxury.
- devalgo 6y ago
  Well that definitely speaks more to the field in general and not necessarily to "beginners" who tune a pretrained model and post the results to arXiv. There's massive incentives to post papers with incremental progress when there's potentially billions of dollars of Grants and VC money for folks with their names on those papers.

gumby 6y ago

I consider this equivalent to the democratization of web site development and then app development. It’s certainly lead to an explosion of crummy and security-nightmare apps but in exchange has been an on-ramp for some good developers and exciting products as well. The massive hype surrounding anything “AI” has caused the literature to become a dumpster fire, yes, but a handful of good papers still appear. Just use a low pass filter as with most things these days.

Datenstrom 6y ago

I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then: 1. Read "Artificial Intelligence A Modern Approach" and complete all the exercises [1] 2. Complete a ML course I recommend the Andrew Ng Coursera one [2] 3. Complete a DL course I recommend the Andrew Ng Coursera one [3] 4. Complete specialization courses in whatever you are interested in such as computer vision, reinforcement learning, natural language processing, etc. These will cover older traditional methods in the introduction also which will be very useful for determining when DL is not the correct solution. Additionally I suggest a data science course which usually covers other important things like data visualization and how to handle bad/missing data. Also, I learned a lot by simply being surrounded by brilliant people who know all this and being able to ask them questions and see how they approach problems. So not really self taught as much as untraditionally taught. Unfortunately not a single person has actually followed the advice. Everyone has only watched random youtube bloggers and read blogs. Some have gotten into trouble after landing a job by talking buzzwords and asked for help but my advice does not change. It does make it rather hard to find a job without a degree though, I would not recommend it. All of mine only come from strong references I have from luckily getting my foot in the door initially. [1]: http://aima.cs.berkeley.edu/index.html http://aima.cs.berkeley.edu/index.html [2]: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning [3]: https://www.coursera.org/specializations/deep-learning https://www.coursera.org/specializations/deep-learning Edit: formatting, typo

gas9S9zw3P9c 6y ago
Thanks for sharing your learning path and congrats to your success. There are so many great resources out there that it's possible for anyone to become an expert. Unfortunately, people like you who are willing to put in the hard work seem to be the minority. All of your success is well-deserved and props to you. Just like you said, most gravitate towards the easy-to-understand videos and blogs instead of confronting their gaps in knowledge. I've had the same experience with giving advice - anything that looks like it requires focused work (solving textbook problems!) or is unsexy is readily ignored in favor of the latest hype demo or high-level framework. > It was primarily driven by the need to solve problems for autonomous systems I was creating. I wonder if your success also had something to do with the fact that you had a specific problem you were trying to solve. Did you feel that your specific problem put everything you were learning into context and made it easier to stay motivated?

mumbisChungo 6y ago

>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. This seems extremely short-sighted to me. If the barrier to entry (on an already relatively young technology) has come down so far that there are a bunch of noobs running rampant right now, does that not bode extremely well for future advancements in the field (assuming some non-zero conversion rate from noobs to productive members of the field over time)? Also, let's not pretend that people with relevant degrees don't make shitty contributions to arXiv all the time.

abnry 6y ago
I'm surprised anyone would default to trusting arXiv papers, or even browse new additions, without first checking up on the credentials and experience of the authors. If the problem is finding new papers, then you really should be using sources other than arXiv.

sdinsn 6y ago

> At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. Note that this is not a problem exclusive to AI or Computer Science. I have a hobbyist interest in Entomology, and I was so disappointed to see that people are still pumping out papers that are minor tweaks off old population modeling papers from the 1980s. The field is shockingly stagnant. I've read random PhD theses from the 70s that are written on damn typewriters that are higher quality than modern papers from so-called "top tier" research universities.

AlexanderNull 6y ago

Yeah, if you're not programming your models in binary then gtfo imposters!! On a more serious note, yes understanding this and anything really takes time and investment. The problem for me at least originally not only with ML but originally with engineering back when I was trying to learn (and couldn't afford school), was finding quality sources for getting started in that process of learning. By providing simplified resources like this google one, the hope is that many beginners can get that one "aha!" moment where they start the basic understanding that allows them to start tinkering and learning. People without a decent understanding shouldn't be submitting research papers, full on agree there. It's basically a waste of everyone's time and harmful for the field of research as a whole as it dilutes the overall signal to noise ratio. However there's so much space in the ML world that doesn't involve research, not only for fun hobby projects, but also even professionally. Resources like this are critical to reducing the knowledge gap out there between researchers and the programmers in the field that work on little ML projects for doing things like sentiment analysis for their company. These sub-research projects are mission critical at a lot of companies yet are held up at the majority of non-FAANG companies because there's only one data scientist while the teams of engineers are clueless as to how to assist.

abhgh 6y ago

I share your sentiment to a fair extent and I have written about it before here on HN. Yes, for a lot of this stuff you don't need a PhD (and frankly I find that marketing weird for the above course). But you do need strong intuitions, understanding of some CS, and math "savviness" i.e. you don't have to know All The Math now, but you should be able to pick up stuff as needed, when you're trying to understand your problem and/or structuring your solution. One could learn all of this stuff online today - the amount of good resources out there is crazy. Frankly, I am jealous, because I began working on ML more than 10 yr ago, and we were relatively starved for resources on pretty much all fronts: resources to study from (reading material or videos), affordable compute power, s/w libraries. But unfortunately, despite their abundance today, most people don't take the time to dive deep. Of all the years of me suggesting courses and books to people (when asked), only ONE (or maybe two) person managed to go through them to a fair extent. But there is a significant fraction of the rest, for which reductive messaging like "become a pro in AI in 3 weeks" has been misleading. As a hiring manager sometimes they are as surprised as me, when an interview doesn't go well, after the resume seemed promising to both sides. And to be very clear, I don't blame them (sure, there are some pretentious opportunists, who flat out lie, but I've found them not to be the norm); all this messaging seems to have created a bubble where often you don't know what you don't know. It's amusing that thrice, rejected candidates reached out me saying that the interview was quite eye opening! I have been at the receiving end too - where 90% of the interview seemed to be about some very specific setting of a library or a method, because that was the conception of ML the interviewer had. I think people should learn, by whatever means, and create stuff because they can - this is the best kind of learning. Silly projects are great too, if they are fun - if they don't advance your understanding, they might motivate you to be less silly! What's missing in this ecosystem is honest messaging about where your skill levels really are. I don't know how to fix it in a way that also doesn't harm, in some way, the widespread learning/awareness reg ML. On a smaller scale though, I have accepted that this has increased my scope of work in screening resumes: if someone lists her github repo, or an arxiv paper, I actually need to spend time to go through them. I don't see this as noise, but a widening of the spectrum of available ML skills in the market; and I need to put in some effort to place an applicant in this spectrum. I've accepted that this is the flipside of working in a hot area: for the multiple job opportunities accessible to me, I have to put in more thought for hiring. I can't have the luxury of the former without the responsibility of the latter. Although, being lazy, I'd totally want to ;).

citilife 6y ago

I am one of those without a PhD, but have taken the time to the learn the math & contribute quite a bit to this area. That being said, I also don't think deep learning itself is not really a "science". The issue I have is you can't predict if a network will learn. We're effectively testing deep learning networks the same way the Romans used to test bridges. Send a bunch of elephants over them, if it holds it's good enough. There's obviously some indicators of success, but on a whole the overall interaction between components is very difficult to calculate and near-impossible to predict. While I think it's important to understand how layers interact and how a given function will impact your optimization, etc. it's not fully required to have a deep understanding of the mathematics, at least for most cases. I also personally don't view anything on arXiv worth anything. I typically will read articles/papers myself if reviewing a candidate and / or would like to see their publications at conferences or journals. Otherwise, it's essentially a blog post (which IMO is fine, but will require me to review it).

andrewprock 6y ago
The mathematics of neural networks is very well understood from a theoretical and scientific perspective. It is easy to say that a neural network will have predictive power given labelled training data. Whether or not the predictions it is making satisfy unstated requirements is another problem altogether.
- deleted 6y ago
  [deleted]
dumb1224 6y ago
> We're effectively testing deep learning networks the same way the Romans used to test bridges. Send a bunch of elephants over them, if it holds it's good enough. Well for human level tasks maybe, but what about other areas of research where we like to discover patterns not seen by a human? Like finding links between genomic interplay with external perturbation such as radiotherapy? It'll be very lucky to have such 'elephants' to test at all.
- 6y ago

zerr 6y ago

Maybe stopping using "AI" in favor of "Statistics" would help?

thallavajhula 6y ago

Isn't that the case with any aspect of Computer Science in general?

deleted 6y ago

[deleted]

Razengan 6y ago

> the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. Then the problem isn't the low barrier and ease of use, it's with arXiv's filtering and how resumes are valued and verified.