10 ms·

I think this field is suffering from some confusion of terminology. In my mind there are three subfields that are crystallizing that each have different goals a

• sidlls 10y ago
  It's also suffering from hype. And the criticism you note isn't one-directional in the field at large. I'm finding that ML/AI researchers deriding ML/Data engineers and "scientists" as not doing "real" ML or AI is becoming a thing, similar to how some computer scientists deride engineering as not doing real computing.
  • WhitneyLand 10y ago
    I hate this. Remember they did it to Michael Faraday in 1813, deriding him as not a real scientist as he was about to play a part in advancing one of the greatest challenges of mankind (unification).
  • logicallee 10y ago
    It is not suffering from hype. There is too little hype. People are vastly underestimating what is about to happen. See my comments here: https://news.ycombinator.com/item?id=13079598 https://news.ycombinator.com/item?id=13079598 under our recent article " Artificial Intelligence Generates Christmas Song". Basically, if there is no pixie dust that makes humans intelligent, and instead it is a matter of the architecture of the brain and the first few years of supervised sensory input, then neural net breakthroughs (which use a similar architecture/topology) have the potential at any moment in time to break through and match general human intelligence. What I mean is that if someone sent back source code from 80 years from now, but we had to run it on a bunch of Amazon / Google servers in a server farm, we're pretty much guaranteed to have enough computing power to do so! (This is a combination of the number of neurons, their number of connections, and their very slow speed.) We have the hardware. Now: we do not actually have the source code from 80 years from now that we can go ahead and run on those machines. So, we're at like heavier-than-air travel right before the Wright Brothers flew at Kittie Hawk. Except we have like jet engines already -- just no way to design them into something that flies. I think that AI is vastly underrated and I watch with incredible interest every single breakthrough. For the second category above, ML built into products or engineering solutions, Alphago surprised me, because Go had an intractibly large possibility space, it's not in any sense subject to brute-forcing or exhaustive search. Dragon's Natural dictation surprised me in that using its model its able to get basically perfect dictation. I've never worked as a transcriptionist, but a small search reveals it has basically annihilated the industry of medical transcription. These are not the big, general breakthrough. But the big, general breakthrough is right there, somewhere. The results researchers are coming up with are astounding, and they're doing it in many cases with neural nets, quite similar to the wiring of the human mind. The hype is faaaaaaaaaaaaaaar less than warranted for the stage that we're at. At any moment someone can put together something that achieves higher-level intelligence and can be set loose upon the world corpus of culture. True, there are no clear indications that this is about to happen (For example: people do not extract innate language algorithms from the human genome which encodes them), so we are not exactly taking many steps that we could be trying to. As far as I know we're not even genetically engineering people to see what different parts of DNA do -- which is obviously a very, very good thing, and who would allow anyone to bear to term a child made as an experiment to see what DNA does. But despite not going from a human starting point, the results that we are achieving in many cases match and surpass human ability - while we do know that in many cases some of their architecture is similar. I feel quite strongly that we have more than enough hardware for general intelligence - and I see advances every day that could end up going past the ppint of no return on it. -- EDIT: got a downvote, but I would prefer a discussion if you think I'm wrong.
  • cma 10y ago
    >It's also suffering from hype. I'm imagining someone tweeting at Robin, several years back: "Good Prediction Markets expert says: Most firms that think they want advanced Prediction Markets really just need linear regression on cleaned-up data." ;)
  • ma2rten 10y ago
    It's also suffering from hype. If you mean hype in media and general public, I agree with you. Research is inherently risky and uncertain, often that is not conveyed correctly. Also research results are often oversold. If you mean that big tech companies are overinvesting in Machine Learning then I have to disagree. It's not a coincidence that the companies that invested the most in Machine Learning (Google and Facebook) are companies that have a lot of data and already use Machine Learning in their products. It provides things like better feed ranking and better signals for search. These "invisible" improvements to existing products are often overlooked. And the criticism you note isn't one-directional in the field at large. Yes, like I mentioned I think there is confusion about terminology. Many ML Engineers (in my definition) call themselves Data Scientists. This leads to misunderstandings, when people don't recognize that others have different goals.
    • unscaled 10y ago
      You mean, some tech companies understand the difference between Data Science, AI in general, ML and Deep Learning and try to utilize the best tools in hand to handle their data. But what is driving this hype, as well as the Blockchain hype, is not engineer-driven companies like Google, but rather MBA-driven buzzword-friendly tech companies (imagine Balmer-era Microsoft), non-tech companies who want to share the cake, the media and finally the investors who are misled by the rest, but end up creating a capital-based feedback loop for them. As far as the people driving the hype train right now are concerned, Machine Learning is the way you do AI, and Deep Learning is just a more powerful (ahem deeper) version of Machine Learning. This is what AlphaGo used to defeat Lee Sedol, so it's obviously superior, and we should use it to process all data, in the same way we should strive to store everything on a blockchain, which is clearly superior to hash tables and databases.
  • shas3 10y ago
    This is the first time I'm hearing that computer scientists deride engineering as not-real-computing. Any references? It doesn't check out based on my background. In fact anecdotally, I've heard the reverse. I've heard EE and stat algorithms folks criticize CS ML/CV researchers for using algorithms as black boxes compared to the rigorous standards of EE/Stat (aka reviewer standards in IEEE Transactions in Information Theory, IEEE Trans Sig Proc, majority of stat journals). Now obviously there are a vast number of CS researchers who are competent at stat-level rigor. But there are also a large number of ML researchers (applied ML?) in CS departments who use algorithms as black boxes.
    • liviu- 10y ago
      My understanding is that the grandparent comment was referring to _software_ engineers.
    • swordswinger12 10y ago
      The prof of my applied ML class this semester jokingly refers to ML research's approach to statistics as "punk stats", which I find to be pretty accurate.
  • c3534l 10y ago
    This and the above comment kind of says my thoughts on the matter. I think there is a very real emerging field that combines mathematics, traditional computer science, and AI into a big family resemblance of quantitative methods and research, but that we're still in the early stages and we're still sorting things out. I remember hearing someone on hacker news joke that when he got out of school people called him a statistician, then people started saying he was a data miner, and now he sells himself as a data scientist, but he hasn't actually done anything differently apart from just keeping up with the field. Sometimes it feels as if we're just playing with words and people get excited for advances and what it means for the future, but the reality is people just keep plodding along make incremental progress, finding new ways to leverage the tools they have, and continuing to publish papers. For all of the tech industry's worship of "disruptive innovation" and massive changes, the reality is that we've been developing and refining our quantitative abilities and figuring out automation since the industrial revolution. AI may become less exciting to people 5 years from now, but we will still continue doing what we've been doing.
• pesenti 10y ago
  Not a bad way to put it. You could make his argument valid though by discussing the economic value of the problems solved by the first category of people vs. the third. Right now I agree with him that the first is bigger than the third. But I believe that balance is starting to tip and the potential value of the third will keep increasing relatively to the first to the point of dwarfing it (hence no bust).
  • akiselev 10y ago
    I don't think that there is a "balance" to tip, it's just that the previously theoretical work of the latter group is starting to become practical to the former in a business setting. Neural networks have been a topic of research for many decades before consumer demand for graphical video games drove the technology to make them practical. Backpropagation was first described as a solution to efficiently training multi-layer neural networks in 1975 but only in the last decade have we developed the infrastructure for non-CS researchers to use complex neural networks using a sub-$1000 teraflop GPU and an iPython notebook. Off the top of my head the closest analogy would be number theory, especially the study of prime numbers. Before information technology in general, number theory was esoteric and considered useless by many pure and applied mathematicians (I'm simplifying a bit for argument's sake) but all of that accumulated research proved massively useful once we started to communicate electronically. WWI and II cryptographers didn't become or absorb number theorists as a group, they just adapted the knowledge to their field under the umbrella of electronic warfare. I think this is happening with data scientists, who are starting to experiment with ML but it's still just another tool in their toolbox. The media hype train focuses on the flashy AI contests and muddles the terminology but the real work [1] is happening behind the scenes in data science. [1] By "real work" I mean work that directly translates into value on a company balance sheet. The theoretical work is important in and of itself and has been happening for decades.
• kevinchen 10y ago
  > I feel like this post is essentially someone from the first group criticizing the last group, saying their methods are not applicable to him. Probably more accurate to say that it's the first group criticizing others in the first group who try to act like people in the third group. Data scientists who use deep learning for everything, when a more interpretable model would do just as well.
  • kennethcwilbur 10y ago
    Or, someone in the first group criticizing observers who fail to distinguish among groups before making predictions.
  • reacharavindh 10y ago
    Sounds very familiar in pattern to those people who were part of the 'big data' hype who would attempt to use Hadoop for everything.. even work that could have been accomplished with an Excel sheet and an hour. Looks like us Engineers have a common pattern in running towards the next shiny thing ;-) Somewhere along the journey/career some of us wise up and learn when to say No, and use the right tools for the job, and ignore the "cool" factor.
• bogomipz 10y ago
  This was a really nice breakdown, thanks. You mentioned that that Deep learning was a method of choice for AI researched because Deep it has unlocked a lot of new application. I have a question - is it also a "method of choice" for researchers because its not well understood yet why Deep Learning actually works?
  • dquail 10y ago
    Good question. Just one anecdotal data point here ... So take it for what it's worth. I'm a grad student focusing on reinforcement learning but have a lot of interaction with many deep learning folk. I'd have to say that they seem mostly motivated to learn how to make deep learning even more powerful and how to apply it. Not so much solving what's going on inside the box.
    • visarga 10y ago
      We do understand in general, just not every neuron and input weight in particular, because they are too many for limited human working memory to hold at once. An automated labyrinth could be solved by hand up to a size, but if it becomes larger, it would become "impossible to understand". We'd need to rely on computers to find the route for us. It's not computer magic, just an ability to hold more data at once. We're limited to 7-8 things - try to remember a string of more digits and see if you can - it's hard, we just have that kind of limitation. So any computer algorithm that can't be broken down into 7-8 understandable parts is hard to grasp. Of course we can make ML tools synthesize the preferred input of any neuron. That does help a little.
      • nharada 10y ago
        For many researchers, "understand" relates more to the underlying math. Things like "what kind of convergence properties and guarantees are there" and "formality, what can this model learn and not learn"?
• ktRolster 10y ago
  A lot of standard automation is being called 'AI' too, because that sells.
• HemanHeartYou 10y ago
  Articles like this show because people are given the title Engineers.