11 ms·

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ fiel

by jstx1 5y ago

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI - I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. - That doesn't make me a skeptic towards the current state of machine learning though. ML doesn't need to lead to general AI. It's already useful in its current forms. That's good enough. It doesn't need to solve all of humanity's problems to be a great tool. I think it's important to make this distinction and for some reason it's left implicit or it's purposefully omitted from the article.

shreyshnaccount 5y ago
I'm in favor of changing the terminology from AI and ML to something along the lines of 'prediction model' so that the idea of machines 'thinking' is replaced with them 'predicting'. it's just easier for our mushy meat brains to think that AI and ML means that it'll lead to general AI or as I like to call it 'general purpose decision maker'. it's all about the language!
- coddle-hark 5y ago
  I like the term “data driven algorithm“. It makes it clear to everyone involved that what we’re doing is just adjusting an algorithm based on the data we have. No-one in their right minds would confuse that with building a true “A.I.”.
  - spockz 5y ago
    What about “data derived algorithm”? The algorithm itself isn’t really driven by data after it has been designed anymore.
    - skohan 5y ago
      I mean if we want to be really accurate, we could say something like "highly dimensional data-derived function"
      - shreyshnaccount 5y ago
        why stop at that? 'high dimensional matrix parameterised data derived non linear function optimisation and unique hypothesis generation' just rolls off the tongue doesn't it xD
  - tw04 5y ago
    To be frank: that very much does not make it clear to everyone involved. If you told the average Joe you had a “data driven algorithm” instead of “AI” you would likely get a blank stare in return.
    - deleted 5y ago
      [deleted]
    - shreyshnaccount 5y ago
      confusion is better than wrongful understanding?
  - falcor84 5y ago
    I'm sorry to say that I don't see any clear line separating "data driven algorithms" from the embodied minds that we are.
  - gmadsen 5y ago
    why? we don't understand the architecture, but the brain certainly uses electrical signals in an algorithmic way
- zoomablemind 5y ago
  In not so long past, there was another popular expression - "computer-aided ...", which was quite fit for the practical use (like CAD for design, CAT for translation etc) Perhaps, CAI for inference or insight would express it more fairly. Alternatively, AI could've stood for 'automated inference', but sure it's all too late to rebrand. We humans still not clear about nature of our own intelligence, yet already claimed being able to manufacture it.
  - skohan 5y ago
    I think inference isn't the right term either. I think current ML is more like automated inductive reasoning.
    - aidenn0 5y ago
      Automated inductive reasoning sounds a lot like artificial intelligence to me...
      - skohan 5y ago
        Idk maybe it's semantics, inference to me sounds more like a logical leap is happening, whereas in my mind the simplest form of inductive reasoning is just expecting a pattern to repeat itself.
        aidenn0 5y ago
        Expecting a pattern to repeat itself may not be sufficient to count as intelligence, but general purpose pattern recognition certainly seems to fit the bill.
        bississippi 5y ago
        Computer Aided Pattern Recognition sounds reasonable in setting public expectations.
- kzrdude 5y ago
  ML seems to be an ok term to me? It's the "intelligence" part in AI that needs a disclaimer.
- MR4D 5y ago
  I propose “heuristic optimization”.
  - dgb23 5y ago
    I like this the most, but you can also generate things, which isn’t implied by optimization strongly.
- esfandia 5y ago
  We already have "Pattern Recognition", not sure why it got absorbed by Machine Learning (the two terms seemed to co-exist with some overlap on what they covered), and then ML got absorbed by AI.
  - jstx1 5y ago
    ML is still widely used and is much more common than AI as a term. So I wouldn't say that it has been absorbed by AI but their use sometimes overlaps depending on the target audience.
- akomtu 5y ago
  Iirc, predictive coding is a well known branch of math that's said to be the next big step towards AI.
- JohnJamesRambo 5y ago
  Do I think or predict?
  - quickthrower2 5y ago
    I predict therefore I will be
- piokoch 5y ago
  When I have encountered for the first time ML term I decided to learn what is that new great stuff. To my great surprise this was a typical old new thing called "statistical inference" in the days when I was working as a statistician. There were a few new things, like ignoring model, choosing right variables, whatever was available was thrown into the equation, if it was clear that such "model" is over-fitted, there were some methods to overcome this by adding some random coefficients to the model that were smoothing it a little. So, the naming is there... could be modified by adding some clarification that we don't care that much about understanding model we plan to use.
darkwater 5y ago
> I think it's important to make this distinction and for some reason it's left implicit or it's purposefully omitted from the article I beg to disagree. They clearly state your opinion at the end of the piece, using the metal-beat analogy. Great things were done by blacksmiths beating metal, but not an ICE
bhntr3 5y ago
> I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI > I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. This isn't how science works though. Quoting the wikipedia page for Thomas Kuhn's "The Structure of Scientific Revolutions" (https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...): "Kuhn challenged the then prevailing view of progress in science in which scientific progress was viewed as "development-by-accumulation" of accepted facts and theories. Kuhn argued for an episodic model in which periods of conceptual continuity where there is cumulative progress, which Kuhn referred to as periods of "normal science", were interrupted by periods of revolutionary science." I think this is the accepted model in the philosophy of science since the 1970s. That's why I find this argument about AI so strange, especially when it comes from respected science writers. The idea that accumulated progress along the current path is insufficient for a breakthrough like AGI is almost obviously true. Your second point is important here. Most researchers aren't concerned with AGI because incremental ML and AI research is interesting and useful in its own right. We can't predict when the next paradigm shift in AI will occur. So it's a bit absurd to be optimistic or skeptical. When that shift happens we don't know if it will catapult us straight to AGI or be another stepping stone on a potentially infinite series of breakthroughs that never reaches AGI. To think of it any other way is contrary to what we know about how science works. I find it odd how much ink is being spent on this question by journalists.
- GeorgeTirebiter 5y ago
  This seems akin to Asimov's "Elevator Effect": https://baixardoc.com/preview/isaac-asimov-66-essays-on-the-past-present-future-5c8d5ce31fd1c https://baixardoc.com/preview/isaac-asimov-66-essays-on-the-... starting p 221. I agree that one would think that Science Fiction writers would have enough of an imagination to be able to consider alternate futures (Cory CYA's by saying such a scenario would make a good SF story) - but there are already promising approaches to AGI: Minsky's "Society of Mind", Jeff Hawkins' neuro-based approaches, the fairly new Hinton idea GLOM: https://www.technologyreview.com/2021/04/16/1021871/geoffrey-hinton-glom-godfather-ai-neural-networks/ https://www.technologyreview.com/2021/04/16/1021871/geoffrey... . “By 2029, computers will have human-level intelligence,” Kurzweil said in an interview at SXSW 2017. Time to get to work, eh? https://www.timeanddate.com/countdown/to?msg=Kurzweil%20AGI%20Working&year=2029&month=12&day=31&hour=23&min=49&sec=59&fromtheme=generic https://www.timeanddate.com/countdown/to?msg=Kurzweil%20AGI%...
SavantIdiot 5y ago
I'm am both. Why I'm pro-AI: Neural nets. I worked on object detection for several years at one company using traditional methods, predating TensorFlow by a few years. We had a very sophisticated pipeline that had a DSP front end and a classical boundary detection scheme with a little neural net. The very first SSDMobileNet we tried blew away 5 years worth of work with about two weeks of training and tuning. Other peers of mine work in industrial manufacturing, and classification and segmentation with off the shelf NN's has revolutionized assembly line testing almost overnight. So yes, DNNs absolutely do some things vastly better than previous technology. Hand's down. Why I'm Anti-AI: hype The class of problems addressed by recent developments in NN/DNN software have failed horribly in scaling to even modestly real-world, rational multi-tasking. ADAS level 5 is the poster child. When hype master Elon Musk backs away, that is telling. We're on the bleeding edge here, IMHO we NEED to try everything. There's no telling which path has fruit. Look at elliptic curves: half a century with no applications, now they are the backbone of the internet. Yes, there will be BS, hype, snake oil, vaporware, but there will also be some amazing tech. I say be patient and skeptical.
jmull 5y ago
> it's left implicit or it's purposefully omitted from the article It's explicitly right there in the essay... > Machine learning has bequeathed us a wealth of automation tools that operate with high degrees of reliability to classify and act on data acquired from the real world. It’s cool! > Brilliant people have done remarkable things with it. You seem to be in agreement with the article but don't realize it.
wffurr 5y ago
Isn't that what's meant by "admittedly impressive"?
skohan 5y ago
Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epigenetics has an effect on information processing. The idea that we could reach some approximation of "general intelligence" by just scaling up some very large matrix operations just seemed like a complete joke. However, as you say, that doesn't mean what we've done in ML is not worthwhile and interesting. We might have over-reached thinking ML is ready to drive a car without major fourth-coming advancements, but use-cases like style transfer and DLSS 2 are downright magical. Even if we just made marginal improvements in current ML, I'm sure there is a ton of untapped potential in terms of applying this tech to novel use-cases.
- fossuser 5y ago
  I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e... Also check out GPT-3’s performance on arithmetic tasks in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63 Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that and that’s just from a relatively dumb predict what comes next model. It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure?
  - skohan 5y ago
    I've heard this airplane argument before, and while I do consider it plausible that AGI might be achievable with some system which is fundamentally much different than the human brain, I still don't think it can be achieved using simple scaling and optimization of the techniques in use today. I think this for a couple reasons: 1. The current gap in complexity is so huge. Nodes in an ANN roughly correspond to neurons, and the brain has somewhere on the order of 100 billion of them. Even if we built an ANN that big, we would only be scratching the surface of the complexity we have in the brain. Each synapse is basically an information processing unit, with behavioral characteristics much more complicated than a simple weight function. 2. The brain is highly specific. The structure and function of the auditory cortex is totally different to that of the motor cortices, to that of the hypothalamus and so on. Some brain regions depend heavily on things like spike timing and ordering to perform their functions. Different brain regions use different mechanisms of plasticity in order to learn. Currently most ANN's we have are vaguely inspired by the visual cortex (which is probably why a lot of the most interesting things to come out of ML so far have been related to image processing) and use something roughly analogous to net firing frequency for signal processing. I would consider it highly likely that our current ANNs are just structurally incapable of performing some of the types of computation we would consider intrinsically linked to what we think of as general intelligence. To make the airplane analogy, I believe we're probably closer to Leonardo da Vinci's early sketches of flying machines than we are to the Right Brothers. We might have the basic idea, but I would wager we're still missing some of the key insights required to get AGI off the ground. edit: it looks like you added some lines while I was typing, so to respond to your last points: > it’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? I totally agree that it's hard to predict, that technology usually advances faster than we expect, and that tremendous progress is being made. But the road to understanding human intelligence has been characterized by a series of periods of premature optimism followed by setbacks. For instance, in the 20th century, when dyes were getting better, and we were starting to understand how different brain regions had different functions, it may have seemed like we were close to just mapping all the different pieces of the brain, and that completing the resulting puzzle would give a clear insight into the workings of the human mind. Of course it turns out we were quite far from that. As far as what we can expect in the years leading up to AGI, I suspect it's going to be something that comes on gradually - I think computers will take on more and more tasks that were once reserved for humans over time, and the way we think about interfacing with technology might change so much that the concept of AGI might not seem relevant at some point. As to whether the current state of things is a failure - I would not characterize it that way. I think we're making real progress, I just also think there is a bit of hubris that we may have "cracked the code" of true machine intelligence. I think we're still a few major revelations away from that.
cratermoon 5y ago
There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/ https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime: https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-arrests-doesnt-protect-people-or-deter-crime/ https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-...
- mgraczyk 5y ago
  It's very easy to fix these problems though. There's nothing inherently broken about the models or direction that prevents error rates from being made more uniform. In fact newer facial recognition models with better datasets do perform approximately equally well across skin tones and sex
  - cratermoon 5y ago
    Easy to fix technically, but first the issue must be recognized and demonstrated, then the delicate process of negotiating the social and economic realities in which the technology operates. And that's the problem with ML in general: its failure to recognize the implicit biases in choice of dataset and training and the resulting problems, of which Microsoft racist chatbot Tay[1] is merely the most blatantly ludicrous. 1 https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
    - mgraczyk 5y ago
      And the first cars didn't have seatbelts. It's fine, these are not complicated problems, and they are much easier to spot and fix than most problems in software engineering at scale. Don't be fooled by the negative PR campaigns and clickbait, there's no reason to be skeptical about ML in general because of this. Also, Tay attempted to solve a much harder problem than image classification. It's hard to build a safe hyperloop. It's no longer hard to build a safe microwave oven.
emrah 5y ago
> It doesn't need to solve all of humanity's problems to be a great tool. As a side note, I'd like to say humanity's own intelligence is actually able to come up with solutions to its problems, we don't need AGI for that. Humanity is unable to implement those solutions for reasons beyond technical. How an AGI would get over those hurdles I have no idea
- hirako2000 5y ago
  Humanity has been able to introduce at last as many problems as it managed to solve. The big question is what is AI/ML trying to accomplish.
js8 5y ago
> I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. I actually think that AGI is deceptively simple. I don't have a proof, but I have a (rather embryonic, frankly) theory of how is it gonna work. I believe AGI is an analogue of third Futamura projection, but for (reinforcement) learners and not compilers. So the first level is you have problem and a learner, and you teach learner to solve the problem. The representation of the problem is implicit in the learner. The second level is that you have a language, which can describe the problem and its solution, and a (2nd level) learner, and you teach the 2nd level learner to create (1st level) solvers of the problem based on the problem description language. The ability to interpret the problem description language is implicit in the 2nd level learner. The third level is, you have a general description language that is capable of describing any problem description language, and you teach the 3rd level learner to take a description of the problem description language, and produce 2nd level learners that can use this language to solve problems created in it. Now, just like in Futamura projections, this is where it stops. You have a "generally intelligent" creature on the 3rd level. You can talk to them on level of how to effectively describe or solve problems (create a specialized language for it) and they will come all the way down with the way to attack (solve) them. In humans, the 3rd level, general intelligence (AKA "sentience"), evolved eventually from the 2nd level, and it was a creation of the general internal language (which probably co-evolved to be shared). The 2nd level is an internal representation of the world that can be manipulated, but only ever refer to the external world, not itself, so it allows creatures to make conscious plans, but lack the ability to reflect on the planning (and also learning) process itself. The "bicameral mind" is a theory how we acquired 3rd level from the 2nd, and the 3rd level is why "we are strange loops". Anway, the problem is, the higher you go up the chain, the harder it becomes to create the learner, it's a lot more general problem. But I think the ladder must be, and should be, climbed. I believe that Deepmind (and RL research) has solved the 1st level, is now working on the 2nd level, but they already somewhat dimly see the 3rd level.

by jstx1 5y ago

shreyshnaccount 5y ago

I'm in favor of changing the terminology from AI and ML to something along the lines of 'prediction model' so that the idea of machines 'thinking' is replaced with them 'predicting'. it's just easier for our mushy meat brains to think that AI and ML means that it'll lead to general AI or as I like to call it 'general purpose decision maker'. it's all about the language!

coddle-hark 5y ago
I like the term “data driven algorithm“. It makes it clear to everyone involved that what we’re doing is just adjusting an algorithm based on the data we have. No-one in their right minds would confuse that with building a true “A.I.”.
- spockz 5y ago
  What about “data derived algorithm”? The algorithm itself isn’t really driven by data after it has been designed anymore.
  - skohan 5y ago
    I mean if we want to be really accurate, we could say something like "highly dimensional data-derived function"
    - shreyshnaccount 5y ago
      why stop at that? 'high dimensional matrix parameterised data derived non linear function optimisation and unique hypothesis generation' just rolls off the tongue doesn't it xD
- tw04 5y ago
  To be frank: that very much does not make it clear to everyone involved. If you told the average Joe you had a “data driven algorithm” instead of “AI” you would likely get a blank stare in return.
  - deleted 5y ago
    [deleted]
  - shreyshnaccount 5y ago
    confusion is better than wrongful understanding?
- falcor84 5y ago
  I'm sorry to say that I don't see any clear line separating "data driven algorithms" from the embodied minds that we are.
- gmadsen 5y ago
  why? we don't understand the architecture, but the brain certainly uses electrical signals in an algorithmic way
zoomablemind 5y ago
In not so long past, there was another popular expression - "computer-aided ...", which was quite fit for the practical use (like CAD for design, CAT for translation etc) Perhaps, CAI for inference or insight would express it more fairly. Alternatively, AI could've stood for 'automated inference', but sure it's all too late to rebrand. We humans still not clear about nature of our own intelligence, yet already claimed being able to manufacture it.
- skohan 5y ago
  I think inference isn't the right term either. I think current ML is more like automated inductive reasoning.
  - aidenn0 5y ago
    Automated inductive reasoning sounds a lot like artificial intelligence to me...
    - skohan 5y ago
      Idk maybe it's semantics, inference to me sounds more like a logical leap is happening, whereas in my mind the simplest form of inductive reasoning is just expecting a pattern to repeat itself.
      - aidenn0 5y ago
        Expecting a pattern to repeat itself may not be sufficient to count as intelligence, but general purpose pattern recognition certainly seems to fit the bill.
        bississippi 5y ago
        Computer Aided Pattern Recognition sounds reasonable in setting public expectations.
kzrdude 5y ago
ML seems to be an ok term to me? It's the "intelligence" part in AI that needs a disclaimer.
MR4D 5y ago
I propose “heuristic optimization”.
- dgb23 5y ago
  I like this the most, but you can also generate things, which isn’t implied by optimization strongly.
esfandia 5y ago
We already have "Pattern Recognition", not sure why it got absorbed by Machine Learning (the two terms seemed to co-exist with some overlap on what they covered), and then ML got absorbed by AI.
- jstx1 5y ago
  ML is still widely used and is much more common than AI as a term. So I wouldn't say that it has been absorbed by AI but their use sometimes overlaps depending on the target audience.
akomtu 5y ago
Iirc, predictive coding is a well known branch of math that's said to be the next big step towards AI.
JohnJamesRambo 5y ago
Do I think or predict?
- quickthrower2 5y ago
  I predict therefore I will be
piokoch 5y ago
When I have encountered for the first time ML term I decided to learn what is that new great stuff. To my great surprise this was a typical old new thing called "statistical inference" in the days when I was working as a statistician. There were a few new things, like ignoring model, choosing right variables, whatever was available was thrown into the equation, if it was clear that such "model" is over-fitted, there were some methods to overcome this by adding some random coefficients to the model that were smoothing it a little. So, the naming is there... could be modified by adding some clarification that we don't care that much about understanding model we plan to use.

darkwater 5y ago

> I think it's important to make this distinction and for some reason it's left implicit or it's purposefully omitted from the article I beg to disagree. They clearly state your opinion at the end of the piece, using the metal-beat analogy. Great things were done by blacksmiths beating metal, but not an ICE

bhntr3 5y ago

> I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI > I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. This isn't how science works though. Quoting the wikipedia page for Thomas Kuhn's "The Structure of Scientific Revolutions" (https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Revolutions https://en.wikipedia.org/wiki/The_Structure_of_Scientific_Re...): "Kuhn challenged the then prevailing view of progress in science in which scientific progress was viewed as "development-by-accumulation" of accepted facts and theories. Kuhn argued for an episodic model in which periods of conceptual continuity where there is cumulative progress, which Kuhn referred to as periods of "normal science", were interrupted by periods of revolutionary science." I think this is the accepted model in the philosophy of science since the 1970s. That's why I find this argument about AI so strange, especially when it comes from respected science writers. The idea that accumulated progress along the current path is insufficient for a breakthrough like AGI is almost obviously true. Your second point is important here. Most researchers aren't concerned with AGI because incremental ML and AI research is interesting and useful in its own right. We can't predict when the next paradigm shift in AI will occur. So it's a bit absurd to be optimistic or skeptical. When that shift happens we don't know if it will catapult us straight to AGI or be another stepping stone on a potentially infinite series of breakthroughs that never reaches AGI. To think of it any other way is contrary to what we know about how science works. I find it odd how much ink is being spent on this question by journalists.

GeorgeTirebiter 5y ago
This seems akin to Asimov's "Elevator Effect": https://baixardoc.com/preview/isaac-asimov-66-essays-on-the-past-present-future-5c8d5ce31fd1c https://baixardoc.com/preview/isaac-asimov-66-essays-on-the-... starting p 221. I agree that one would think that Science Fiction writers would have enough of an imagination to be able to consider alternate futures (Cory CYA's by saying such a scenario would make a good SF story) - but there are already promising approaches to AGI: Minsky's "Society of Mind", Jeff Hawkins' neuro-based approaches, the fairly new Hinton idea GLOM: https://www.technologyreview.com/2021/04/16/1021871/geoffrey-hinton-glom-godfather-ai-neural-networks/ https://www.technologyreview.com/2021/04/16/1021871/geoffrey... . “By 2029, computers will have human-level intelligence,” Kurzweil said in an interview at SXSW 2017. Time to get to work, eh? https://www.timeanddate.com/countdown/to?msg=Kurzweil%20AGI%20Working&year=2029&month=12&day=31&hour=23&min=49&sec=59&fromtheme=generic https://www.timeanddate.com/countdown/to?msg=Kurzweil%20AGI%...

SavantIdiot 5y ago

I'm am both. Why I'm pro-AI: Neural nets. I worked on object detection for several years at one company using traditional methods, predating TensorFlow by a few years. We had a very sophisticated pipeline that had a DSP front end and a classical boundary detection scheme with a little neural net. The very first SSDMobileNet we tried blew away 5 years worth of work with about two weeks of training and tuning. Other peers of mine work in industrial manufacturing, and classification and segmentation with off the shelf NN's has revolutionized assembly line testing almost overnight. So yes, DNNs absolutely do some things vastly better than previous technology. Hand's down. Why I'm Anti-AI: hype The class of problems addressed by recent developments in NN/DNN software have failed horribly in scaling to even modestly real-world, rational multi-tasking. ADAS level 5 is the poster child. When hype master Elon Musk backs away, that is telling. We're on the bleeding edge here, IMHO we NEED to try everything. There's no telling which path has fruit. Look at elliptic curves: half a century with no applications, now they are the backbone of the internet. Yes, there will be BS, hype, snake oil, vaporware, but there will also be some amazing tech. I say be patient and skeptical.

jmull 5y ago

> it's left implicit or it's purposefully omitted from the article It's explicitly right there in the essay... > Machine learning has bequeathed us a wealth of automation tools that operate with high degrees of reliability to classify and act on data acquired from the real world. It’s cool! > Brilliant people have done remarkable things with it. You seem to be in agreement with the article but don't realize it.

wffurr 5y ago

Isn't that what's meant by "admittedly impressive"?

skohan 5y ago

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epigenetics has an effect on information processing. The idea that we could reach some approximation of "general intelligence" by just scaling up some very large matrix operations just seemed like a complete joke. However, as you say, that doesn't mean what we've done in ML is not worthwhile and interesting. We might have over-reached thinking ML is ready to drive a car without major fourth-coming advancements, but use-cases like style transfer and DLSS 2 are downright magical. Even if we just made marginal improvements in current ML, I'm sure there is a ton of untapped potential in terms of applying this tech to novel use-cases.

fossuser 5y ago
I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e... Also check out GPT-3’s performance on arithmetic tasks in the original paper (https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165) Pages: 21-23, 63 Which shows some generality, the best way to accurately predict an arithmetic answer is to deduce how the mathematical rules work. That paper shows some evidence of that and that’s just from a relatively dumb predict what comes next model. It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure?
- skohan 5y ago
  I've heard this airplane argument before, and while I do consider it plausible that AGI might be achievable with some system which is fundamentally much different than the human brain, I still don't think it can be achieved using simple scaling and optimization of the techniques in use today. I think this for a couple reasons: 1. The current gap in complexity is so huge. Nodes in an ANN roughly correspond to neurons, and the brain has somewhere on the order of 100 billion of them. Even if we built an ANN that big, we would only be scratching the surface of the complexity we have in the brain. Each synapse is basically an information processing unit, with behavioral characteristics much more complicated than a simple weight function. 2. The brain is highly specific. The structure and function of the auditory cortex is totally different to that of the motor cortices, to that of the hypothalamus and so on. Some brain regions depend heavily on things like spike timing and ordering to perform their functions. Different brain regions use different mechanisms of plasticity in order to learn. Currently most ANN's we have are vaguely inspired by the visual cortex (which is probably why a lot of the most interesting things to come out of ML so far have been related to image processing) and use something roughly analogous to net firing frequency for signal processing. I would consider it highly likely that our current ANNs are just structurally incapable of performing some of the types of computation we would consider intrinsically linked to what we think of as general intelligence. To make the airplane analogy, I believe we're probably closer to Leonardo da Vinci's early sketches of flying machines than we are to the Right Brothers. We might have the basic idea, but I would wager we're still missing some of the key insights required to get AGI off the ground. edit: it looks like you added some lines while I was typing, so to respond to your last points: > it’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? I totally agree that it's hard to predict, that technology usually advances faster than we expect, and that tremendous progress is being made. But the road to understanding human intelligence has been characterized by a series of periods of premature optimism followed by setbacks. For instance, in the 20th century, when dyes were getting better, and we were starting to understand how different brain regions had different functions, it may have seemed like we were close to just mapping all the different pieces of the brain, and that completing the resulting puzzle would give a clear insight into the workings of the human mind. Of course it turns out we were quite far from that. As far as what we can expect in the years leading up to AGI, I suspect it's going to be something that comes on gradually - I think computers will take on more and more tasks that were once reserved for humans over time, and the way we think about interfacing with technology might change so much that the concept of AGI might not seem relevant at some point. As to whether the current state of things is a failure - I would not characterize it that way. I think we're making real progress, I just also think there is a bit of hubris that we may have "cracked the code" of true machine intelligence. I think we're still a few major revelations away from that.

cratermoon 5y ago

There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/ https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime: https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-arrests-doesnt-protect-people-or-deter-crime/ https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-...

mgraczyk 5y ago
It's very easy to fix these problems though. There's nothing inherently broken about the models or direction that prevents error rates from being made more uniform. In fact newer facial recognition models with better datasets do perform approximately equally well across skin tones and sex
- cratermoon 5y ago
  Easy to fix technically, but first the issue must be recognized and demonstrated, then the delicate process of negotiating the social and economic realities in which the technology operates. And that's the problem with ML in general: its failure to recognize the implicit biases in choice of dataset and training and the resulting problems, of which Microsoft racist chatbot Tay[1] is merely the most blatantly ludicrous. 1 https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
  - mgraczyk 5y ago
    And the first cars didn't have seatbelts. It's fine, these are not complicated problems, and they are much easier to spot and fix than most problems in software engineering at scale. Don't be fooled by the negative PR campaigns and clickbait, there's no reason to be skeptical about ML in general because of this. Also, Tay attempted to solve a much harder problem than image classification. It's hard to build a safe hyperloop. It's no longer hard to build a safe microwave oven.

emrah 5y ago

> It doesn't need to solve all of humanity's problems to be a great tool. As a side note, I'd like to say humanity's own intelligence is actually able to come up with solutions to its problems, we don't need AGI for that. Humanity is unable to implement those solutions for reasons beyond technical. How an AGI would get over those hurdles I have no idea

hirako2000 5y ago
Humanity has been able to introduce at last as many problems as it managed to solve. The big question is what is AI/ML trying to accomplish.

js8 5y ago

> I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. I actually think that AGI is deceptively simple. I don't have a proof, but I have a (rather embryonic, frankly) theory of how is it gonna work. I believe AGI is an analogue of third Futamura projection, but for (reinforcement) learners and not compilers. So the first level is you have problem and a learner, and you teach learner to solve the problem. The representation of the problem is implicit in the learner. The second level is that you have a language, which can describe the problem and its solution, and a (2nd level) learner, and you teach the 2nd level learner to create (1st level) solvers of the problem based on the problem description language. The ability to interpret the problem description language is implicit in the 2nd level learner. The third level is, you have a general description language that is capable of describing any problem description language, and you teach the 3rd level learner to take a description of the problem description language, and produce 2nd level learners that can use this language to solve problems created in it. Now, just like in Futamura projections, this is where it stops. You have a "generally intelligent" creature on the 3rd level. You can talk to them on level of how to effectively describe or solve problems (create a specialized language for it) and they will come all the way down with the way to attack (solve) them. In humans, the 3rd level, general intelligence (AKA "sentience"), evolved eventually from the 2nd level, and it was a creation of the general internal language (which probably co-evolved to be shared). The 2nd level is an internal representation of the world that can be manipulated, but only ever refer to the external world, not itself, so it allows creatures to make conscious plans, but lack the ability to reflect on the planning (and also learning) process itself. The "bicameral mind" is a theory how we acquired 3rd level from the 2nd, and the 3rd level is why "we are strange loops". Anway, the problem is, the higher you go up the chain, the harder it becomes to create the learner, it's a lot more general problem. But I think the ladder must be, and should be, climbed. I believe that Deepmind (and RL research) has solved the 1st level, is now working on the 2nd level, but they already somewhat dimly see the 3rd level.