5 ms·

Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty o

by MiroF 6y ago

Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty obvious to me.

hopfscotch7 6y ago
Yeah, we sure are good at solving the iris data set nowadays.
mellosouls 6y ago
There have been plenty of articles about progress in machine learning, but nothing in genuine (human-like) intelligence. I would expect that's the issue for most people who are interested in the subject.
mehmetoguzderin 6y ago
I think ML naysaying stems mostly from extravagant claims of pop-sci articles around papers or of the authors themselves. It is always good to have some amount of people that point out that the models aren't some cyborg superheroes or has to be a step towards AGI but be more practical.
bsaul 6y ago
Siri still isn’t able to understand « do NOT set the alarm to 3pm », and many image classifier produces aberrations that no human would ever commit. Many people feel that ML has so far only produced « tricks », but still doesn’t show any sign of « understanding » anything. As in, provide meaning. It may be unfair, but i think that at this point people would be more impressed by a program « smiling » at a good joke than by something able to process millions of a positions per second, or « learn » good moves by playing billions of games against itself.
- tikititaki 6y ago
  While I can't speak for Siri, Google's voice recognition continues to surprise and even frighten me. And if you go online and look up deepfakes or AI generated voice of famous people, it also is freakishly accurate. Sure, there are some issues and most of them still fall somewhere in the uncanny valley.. but we are just starting to fully exploit this technology. AI that people don't tangibly see but make a giant impact in their lives is for example the Facebook algorithms that decide what shows up on your feed. When a large subset of the population gets majority of their news from Facbeook, Facebook has effectively created a mechanical system that decides what a large chunk of the population is aware of. I don't think people fully realize what effect that has had on our entire society, including elections. The future will only see more of this type of thing.
  - slykar 6y ago
    I'm always disappointed with Google Asistant. When I say "navigate back home" it tries to route me to a company called "Home". Great job. I've included a hint "back". I've never been to a company called "Home". I'm after work. This is a fun example, but I never get good results. I say "weather" and it shows me the weather, just in a different city. FFS. Maybe in the next 5 years I'll be finally able to do something using voice commands. Right now I just want to smash my phone.
    - tikititaki 6y ago
      Fair enough. Usually when I use "Navigate to the nearest gas station" or "navigate home" or "set an alarm for 8am on Tuesday" it all works perfectly for me, but I concede I don't use the feature that often so perhaps I've just been using it for well-adjusted commands.
    - sdenton4 6y ago
      That's the convolution of two problems: a) Good general-purpose automatic speech recognition, which was an unattainable holy grail for DECADES. Previously, you would have to record a lot of your own voice to train the system for you in particular before it was at all reliable. Now it basically just works. b) Making good use of the results of (a). Your examples are clearly in this category. This is ALSO a very hard problem; voice interfaces are basically brand new creatures, and I expect we'll be seeing 'best practices' form up for a while yet.
- MiroF 6y ago
  Sure, ML is not at human-level intelligence - it is very much a tool-AI where we give a task to the machine and let it get very good at that. Nonetheless, it seems like progress on that front has been made incredibly quickly. Sure, Siri might not always "understand" what you're asking but the ML is able to very accurately transcribe your voice to text, something not really possible a few decades ago. Image classifiers produce misclassifications, but so do humans - it is a very alien technology to us, so we view some of the machine's misclassifications as absurd, but at the same time the machine might view some of our misclassifications as obviously off-base as well. None of what you said seems to explain the consistent negativity on HN to what has really been a transformative technology in a lot of ways.
  - foldr 6y ago
    Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.
    - MiroF 6y ago
      In the past three years we have dropped error rates by a factor of 3, I don’t really think that claim holds water. We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.
      - bildung 6y ago
        Both voice recognition had first working research machines in the 1950s. Of course present models are way better than these, but fundamentally these are the same as those in the '50s, "just" with tremendously better hardware and algorithms. But there is zero intelligence in these, the models have no internal concept of language or the world around them. This will certainly produce many great specific solutions for specific problems, but generic problems like driving a car are most probably not solvable without an internal world model. I'm one of the naysayers and have a background in educational science, I'm constantly baffled why AI research seemingly never looks at theories of human learning (though it may very well be that I haven't looked closely enough!). What essentially all AI approaches do is modelling associative learning (https://en.wikipedia.org/wiki/Learning#Associative_learning https://en.wikipedia.org/wiki/Learning#Associative_learning), but employing more and more processing power for training. This is akin to having a fruit fly brain and copy-pasting that over and over again, in the hope somehow a higher order will organize from that by itself. But most things humans reason about are not learned this way, but rather by infering meaning from things and situations, i.e. one learns how a steering wheel works not through hundreds and hundreds of trial and error cases, but through single Aha! moments in which ad-hoc generated mental models (concepts) are validated in the environment. And GPT-3 has no knowledge organized in a hierarchical system of concepts, just as ELIZA didn't have one.
criddell 6y ago
I wonder if it's because of the name machine learning? It seems a lot of applications boil down to some type of discriminator or pattern matcher.
- MiroF 6y ago
  > It seems a lot of applications boil down to some type of discriminator or pattern matcher This doesn't seem to imply that no learning is going on or that there is some sort of simple process. If I ask you to say whether something is a cat or a dog, you are essentially functioning as a discriminator there - but there is still a lot of process that goes on behind that.
rubber_duck 6y ago
I recently watched a podcast with Stephen Wolfram (few months old) where he runs a photo of himself through Wolfram Alpha "ImageIdentify" and it classifies him as a plunger with 50% probability and 8% probability it's a human [1]. He then goes on saying how he's working on systems that would allow a self driving system decide what to do based on if the collision detected is a human or inanimate object, etc. This was just one particularly ironic example of what I'm seeing with ML - we saw this level of accuracy in classification years ago - where is the enormous progress ? There have been pockets where ML shows it's applicable with encouraging initial results but after that it seems to me that we suck at perfecting those system and the progress has been incremental at best. The only exception I can see is stuff like self-play where the system can generate insane amount of training data. [1] https://youtu.be/ez773teNFYA?t=8420 https://youtu.be/ez773teNFYA?t=8420
- MiroF 6y ago
  Classification accuracy is crazy good, I'm surprised that an obvious photo of him would be viewed as a plunger unless it's an adversarial example. No offense to Wolfram, but they really are hardly at the cutting edge of ML research nonetheless.
pmezard 6y ago
> To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. Maybe because there is a larger fraction of people who know or see how the sausages are made. There are cool things, but is not magical nor transformative. At least not yet.
- MiroF 6y ago
  I work in “making the sausage” - deep learning techniques have already completely transformed my field and are transforming many others. Most of the people I see naysaying don’t appear to causally work in the field, because their critiques are not the same as the ones people in the field are using.
  - pmezard 6y ago
    Can you give a concrete example of deep-learning techniques which have transformed a field? Something not theoretical or confined in a lab? Something which may have changed my life?
    - theferalrobot 6y ago
      Not OP but I can speak with a small example from my previous field - coral reef ecology. It used to be standard to collect video footage and hand annotate small portions of the footage to draw up some statistics about the reef progression. Today we can collect the footage and let ml annotate the entirety of it to get much more accurate and precise statistics about the progression/recession. Not only that but we can do that on an organism basis and also see do highly local analyses. This wasn’t possible 10 years ago. I think the applications for ML outside of consumer tech are far larger than within but everyone here seems focused on that for some reason.

by MiroF 6y ago

Absolutely. To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. The evidence of enormous progress seems pretty obvious to me.

hopfscotch7 6y ago

Yeah, we sure are good at solving the iris data set nowadays.

mellosouls 6y ago

There have been plenty of articles about progress in machine learning, but nothing in genuine (human-like) intelligence. I would expect that's the issue for most people who are interested in the subject.

mehmetoguzderin 6y ago

I think ML naysaying stems mostly from extravagant claims of pop-sci articles around papers or of the authors themselves. It is always good to have some amount of people that point out that the models aren't some cyborg superheroes or has to be a step towards AGI but be more practical.

bsaul 6y ago

Siri still isn’t able to understand « do NOT set the alarm to 3pm », and many image classifier produces aberrations that no human would ever commit. Many people feel that ML has so far only produced « tricks », but still doesn’t show any sign of « understanding » anything. As in, provide meaning. It may be unfair, but i think that at this point people would be more impressed by a program « smiling » at a good joke than by something able to process millions of a positions per second, or « learn » good moves by playing billions of games against itself.

tikititaki 6y ago
While I can't speak for Siri, Google's voice recognition continues to surprise and even frighten me. And if you go online and look up deepfakes or AI generated voice of famous people, it also is freakishly accurate. Sure, there are some issues and most of them still fall somewhere in the uncanny valley.. but we are just starting to fully exploit this technology. AI that people don't tangibly see but make a giant impact in their lives is for example the Facebook algorithms that decide what shows up on your feed. When a large subset of the population gets majority of their news from Facbeook, Facebook has effectively created a mechanical system that decides what a large chunk of the population is aware of. I don't think people fully realize what effect that has had on our entire society, including elections. The future will only see more of this type of thing.
- slykar 6y ago
  I'm always disappointed with Google Asistant. When I say "navigate back home" it tries to route me to a company called "Home". Great job. I've included a hint "back". I've never been to a company called "Home". I'm after work. This is a fun example, but I never get good results. I say "weather" and it shows me the weather, just in a different city. FFS. Maybe in the next 5 years I'll be finally able to do something using voice commands. Right now I just want to smash my phone.
  - tikititaki 6y ago
    Fair enough. Usually when I use "Navigate to the nearest gas station" or "navigate home" or "set an alarm for 8am on Tuesday" it all works perfectly for me, but I concede I don't use the feature that often so perhaps I've just been using it for well-adjusted commands.
  - sdenton4 6y ago
    That's the convolution of two problems: a) Good general-purpose automatic speech recognition, which was an unattainable holy grail for DECADES. Previously, you would have to record a lot of your own voice to train the system for you in particular before it was at all reliable. Now it basically just works. b) Making good use of the results of (a). Your examples are clearly in this category. This is ALSO a very hard problem; voice interfaces are basically brand new creatures, and I expect we'll be seeing 'best practices' form up for a while yet.
MiroF 6y ago
Sure, ML is not at human-level intelligence - it is very much a tool-AI where we give a task to the machine and let it get very good at that. Nonetheless, it seems like progress on that front has been made incredibly quickly. Sure, Siri might not always "understand" what you're asking but the ML is able to very accurately transcribe your voice to text, something not really possible a few decades ago. Image classifiers produce misclassifications, but so do humans - it is a very alien technology to us, so we view some of the machine's misclassifications as absurd, but at the same time the machine might view some of our misclassifications as obviously off-base as well. None of what you said seems to explain the consistent negativity on HN to what has really been a transformative technology in a lot of ways.
- foldr 6y ago
  Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.
  - MiroF 6y ago
    In the past three years we have dropped error rates by a factor of 3, I don’t really think that claim holds water. We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.
    - bildung 6y ago
      Both voice recognition had first working research machines in the 1950s. Of course present models are way better than these, but fundamentally these are the same as those in the '50s, "just" with tremendously better hardware and algorithms. But there is zero intelligence in these, the models have no internal concept of language or the world around them. This will certainly produce many great specific solutions for specific problems, but generic problems like driving a car are most probably not solvable without an internal world model. I'm one of the naysayers and have a background in educational science, I'm constantly baffled why AI research seemingly never looks at theories of human learning (though it may very well be that I haven't looked closely enough!). What essentially all AI approaches do is modelling associative learning (https://en.wikipedia.org/wiki/Learning#Associative_learning https://en.wikipedia.org/wiki/Learning#Associative_learning), but employing more and more processing power for training. This is akin to having a fruit fly brain and copy-pasting that over and over again, in the hope somehow a higher order will organize from that by itself. But most things humans reason about are not learned this way, but rather by infering meaning from things and situations, i.e. one learns how a steering wheel works not through hundreds and hundreds of trial and error cases, but through single Aha! moments in which ad-hoc generated mental models (concepts) are validated in the environment. And GPT-3 has no knowledge organized in a hierarchical system of concepts, just as ELIZA didn't have one.

criddell 6y ago

I wonder if it's because of the name machine learning? It seems a lot of applications boil down to some type of discriminator or pattern matcher.

MiroF 6y ago
> It seems a lot of applications boil down to some type of discriminator or pattern matcher This doesn't seem to imply that no learning is going on or that there is some sort of simple process. If I ask you to say whether something is a cat or a dog, you are essentially functioning as a discriminator there - but there is still a lot of process that goes on behind that.

rubber_duck 6y ago

I recently watched a podcast with Stephen Wolfram (few months old) where he runs a photo of himself through Wolfram Alpha "ImageIdentify" and it classifies him as a plunger with 50% probability and 8% probability it's a human [1]. He then goes on saying how he's working on systems that would allow a self driving system decide what to do based on if the collision detected is a human or inanimate object, etc. This was just one particularly ironic example of what I'm seeing with ML - we saw this level of accuracy in classification years ago - where is the enormous progress ? There have been pockets where ML shows it's applicable with encouraging initial results but after that it seems to me that we suck at perfecting those system and the progress has been incremental at best. The only exception I can see is stuff like self-play where the system can generate insane amount of training data. [1] https://youtu.be/ez773teNFYA?t=8420 https://youtu.be/ez773teNFYA?t=8420

MiroF 6y ago
Classification accuracy is crazy good, I'm surprised that an obvious photo of him would be viewed as a plunger unless it's an adversarial example. No offense to Wolfram, but they really are hardly at the cutting edge of ML research nonetheless.

pmezard 6y ago

> To be perfectly honest, it surprises me the extent to which ML naysaying seems to be popular on HN. Maybe because there is a larger fraction of people who know or see how the sausages are made. There are cool things, but is not magical nor transformative. At least not yet.

MiroF 6y ago
I work in “making the sausage” - deep learning techniques have already completely transformed my field and are transforming many others. Most of the people I see naysaying don’t appear to causally work in the field, because their critiques are not the same as the ones people in the field are using.
- pmezard 6y ago
  Can you give a concrete example of deep-learning techniques which have transformed a field? Something not theoretical or confined in a lab? Something which may have changed my life?
  - theferalrobot 6y ago
    Not OP but I can speak with a small example from my previous field - coral reef ecology. It used to be standard to collect video footage and hand annotate small portions of the footage to draw up some statistics about the reef progression. Today we can collect the footage and let ml annotate the entirety of it to get much more accurate and precise statistics about the progression/recession. Not only that but we can do that on an organism basis and also see do highly local analyses. This wasn’t possible 10 years ago. I think the applications for ML outside of consumer tech are far larger than within but everyone here seems focused on that for some reason.