4 ms·

> The crux of the problem is that the field of artificial intelligence has not come to grips with the infinite complexity of language. I stopped reading at tha

by gok 8y ago

> The crux of the problem is that the field of artificial intelligence has not come to grips with the infinite complexity of language. I stopped reading at that point. The field came to grips with the infinite complexity of language since before it was called "artificial intelligence."

myhf 8y ago
And now that the field has come to grips with the infinite complexity of language, when should we expect the field to act on that understanding instead of attempting to bypass it? https://www.theatlantic.com/technology/archive/2018/01/the-shallowness-of-google-translate/551570/ https://www.theatlantic.com/technology/archive/2018/01/the-s...
Animats 8y ago
Machine translation is getting pretty good, simply from crunching on enough text. The complexity of language looks finite. Translation between all the European languages works fairly well. Asian languages, not so much yet. Strong AI still doesn't work. "Common sense" remains hard. Unstructured manipulation is still not very good. But legged locomotion is much better, as is vision processing. Combining machine learning and geometry has promise. Look at how Waymo does automatic driving. (Not Uber or Tesla; they have no clue how to do it safely, as their crash record demonstrates.) We're way ahead of where things were in the "AI winter", 1985-2005. This time the startups make money and do useful things. Progress will continue because there is revenue. AI used to be tiny - about 20-50 people at MIT, CMU, and Stanford. Now it's at least a thousand times bigger. Progress will be made by brute force. (Me: MSCS, Stanford, 1985. I met most of the greats of classical logic-based AI. Trying to hammer the real world into predicate calculus just doesn't work. The expert systems guys were in denial big-time about this.)
- gugagore 8y ago
  Can you elaborate specially on Waymo's approach of combining machine learning and geometry, or point to a resource?
- seanmcdirmid 8y ago
  > We're way ahead of where things were in the "AI winter", 1985-2005. That was the second AI winter. The first AI winter was in the early/mid 70s. It is not inconceivable that a third AI winter will happen eventually.
  - sgt101 8y ago
    I think that you've got your timing out there. Money started to flow straight after Gulf War 1 in 1992/3 because several folks had build components of the logic systems that were used for organising the logistics (Tate : HTN's; Winston schedulers - although I am hazy about this) What I am clear about is that there was money to be had for AI research after that.
    - seanmcdirmid 8y ago
      The second AI winter is a quote from grandparent, not my own addition.
    - DonHopkins 8y ago
      Sorry to drop in here and interrupt off topic, but you recently wrote a great comment that I missed the first time around, and now can't reply to, and don't know how else to get in touch (consider this "moving along"). It's off-topic but at least vaguely AI related. ;) https://news.ycombinator.com/item?id=14983831 https://news.ycombinator.com/item?id=14983831 That is some fascinating stuff I've never heard before. I would love to discuss it more and ask you some questions! Maybe you could make a posting about the Turing Institute or Lighthill Debate? (There's a wikipedia page about Turing but it's kind of dry and lacking of scandal and palace intrigue.) I worked at the Turing Institute in 1992, and it was an amazing place with many great people, including Arthur van Hoff who developed GoodNeWS/HyperNeWS/HyperLook! Unfortunately I never got the chance to meet Donald Michie (maybe he dropped by but I didn't recognize him), but I know from his reputation what a great guy he was. I did hear a funny story about him: Donald Michie once overheard his secretary telling someone on the phone how to pronounce his name (in a Scottish accent): "It's Donald, as in Duck, and Michie, as in Mouse." He was so pissed he refused to speak to her for a month! ;) For what it's worth, that's a great way to remember how his name is pronounced! When I was there in '92 it was being (mis)run by some upper crust Tory gentleman who specialized in bailing out failing companies. He was so uptight he got pissed off I'd written the address as "North Hangover St." on the white board! Sheez. Instead putting him in charge, they should have just called Old Glory: https://www.youtube.com/watch?v=KXnL7sdElno https://www.youtube.com/watch?v=KXnL7sdElno I've been writing an article (still in draft) about the work I did at the Turing Institute (HyperLook): https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-goodnews-99f411e58ce4 https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g... Fun times! Anyway sorry to interrupt.
- sgt101 8y ago
  Formalisations do break down in the limit, Wittgenstein showed everyone that, but the denial is a deep and powerful choice "he must transcend these propositions and then he will see the world alright." There were engine room troubles for symbolic AI, committed choice meant that reasoners were unpredictable and impractical. We saw 100k improvement of compute change sub symbolic AI, and it has done the same for symbolic, and there have been some really smashing advances in the algorithms. Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what? Probabilistic formalisms and MCMC have changed what we can write down and reason about effectively as well. I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? I was more interested to read the HN reaction! My team has had lots of success creating models using our cluster (in 2014) and then a cluster of GPU's which we bought because we could see that this was the way to go. We haven't gone and spent 100x more since then, and yet we now can see plotted out that the leading teams are doing this. The hardware has not improved at a rate of doubling every 3.4mths, it's hardly improved at all for 12 mths. What's happening is that people are spending money to make waves. This is denial big time. Brute force is a wall, you can climb a few bricks, but the top is unobtainable. There have been some improvements in tree search and there have been some insights in terms of where we are wasting time with optimisations that aren't working but I've always thought that DL is brute force at heart and I think we are where we are with it now, and Alphazero is where we are going to be until someone really sees a new way.
  - YeGoblynQueenne 8y ago
    >> I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? Aye. My reaction to that graph was that this rate of progress can't be sustainable. If industry was throwing good money after bad to find out for how long you can keep ussing bubblesort given more and more compute, before having to develop a better sorting algorithm- that's what it would look like. >> Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what? ASP is interesting. There are symbolic learning techniques that learn answer set programs from data, did you know that?
- YeGoblynQueenne 8y ago
  My apologieis but there is an ocean of misconceptions and a galaxy of misinformation, in that comment. >> The complexity of language looks finite. I have to ask what your defintion of "language" is because by formal language theory the only languages that are finite are, well, finite languages- which are simple than regular languages, which can be descirbed by a regular expression. And I'm pretty sure that nobody has ever managed to write a regular expression that can describe all of human language. >> Translation between all the European languages works fairly well. Asian languages, not so much yet. In fact, translation between "all the European lagnuages" does not work at all well. For limited domains and for languages with lots of examples of translation, it works alright. Stray from that assumption and the results quickly descent into incomprehensible gibberish. >> We're way ahead of where things were in the "AI winter", 1985-2005. This time the startups make money and do useful things. Making money is a measure of progress of the industry- not of the science. >> AI used to be tiny - about 20-50 people at MIT, CMU, and Stanford. That was true back in the '50s, when the field began. In the '80s and '90s, at the hight of GOFAI and expert systems, there were several thousands of researchers working on AI. For example, the 5th Generation Computer project was a massive effort by the Japanese Ministry of International Trade and Industry which encompassed pretty much all of that country's computer industry with huge loads of funding- not to mention the reaction in the West who panicked thinking that the Japanese were aobut to do to their computer industry what they had done to its car industry. You might have heard of the failure of the project- but it took thousands of people a long time to fail. So, no, AI was not tiny, by any sense of the word, for any time after the first years of its birth. >> (Me: MSCS, Stanford, 1985. I met most of the greats of classical logic-based AI. Trying to hammer the real world into predicate calculus just doesn't work. The expert systems guys were in denial big-time about this.) Last week I met a guy with a background in theoretical physics, who works in high-performance computing. Am I now qualified to contribute an opinion about theoretical physics and high-performance computing?
empath75 8y ago
If languages were infinitely complex, humans couldn’t do it either.

by gok 8y ago

myhf 8y ago

And now that the field has come to grips with the infinite complexity of language, when should we expect the field to act on that understanding instead of attempting to bypass it? https://www.theatlantic.com/technology/archive/2018/01/the-shallowness-of-google-translate/551570/ https://www.theatlantic.com/technology/archive/2018/01/the-s...

Animats 8y ago

Machine translation is getting pretty good, simply from crunching on enough text. The complexity of language looks finite. Translation between all the European languages works fairly well. Asian languages, not so much yet. Strong AI still doesn't work. "Common sense" remains hard. Unstructured manipulation is still not very good. But legged locomotion is much better, as is vision processing. Combining machine learning and geometry has promise. Look at how Waymo does automatic driving. (Not Uber or Tesla; they have no clue how to do it safely, as their crash record demonstrates.) We're way ahead of where things were in the "AI winter", 1985-2005. This time the startups make money and do useful things. Progress will continue because there is revenue. AI used to be tiny - about 20-50 people at MIT, CMU, and Stanford. Now it's at least a thousand times bigger. Progress will be made by brute force. (Me: MSCS, Stanford, 1985. I met most of the greats of classical logic-based AI. Trying to hammer the real world into predicate calculus just doesn't work. The expert systems guys were in denial big-time about this.)

gugagore 8y ago
Can you elaborate specially on Waymo's approach of combining machine learning and geometry, or point to a resource?
seanmcdirmid 8y ago
> We're way ahead of where things were in the "AI winter", 1985-2005. That was the second AI winter. The first AI winter was in the early/mid 70s. It is not inconceivable that a third AI winter will happen eventually.
- sgt101 8y ago
  I think that you've got your timing out there. Money started to flow straight after Gulf War 1 in 1992/3 because several folks had build components of the logic systems that were used for organising the logistics (Tate : HTN's; Winston schedulers - although I am hazy about this) What I am clear about is that there was money to be had for AI research after that.
  - seanmcdirmid 8y ago
    The second AI winter is a quote from grandparent, not my own addition.
  - DonHopkins 8y ago
    Sorry to drop in here and interrupt off topic, but you recently wrote a great comment that I missed the first time around, and now can't reply to, and don't know how else to get in touch (consider this "moving along"). It's off-topic but at least vaguely AI related. ;) https://news.ycombinator.com/item?id=14983831 https://news.ycombinator.com/item?id=14983831 That is some fascinating stuff I've never heard before. I would love to discuss it more and ask you some questions! Maybe you could make a posting about the Turing Institute or Lighthill Debate? (There's a wikipedia page about Turing but it's kind of dry and lacking of scandal and palace intrigue.) I worked at the Turing Institute in 1992, and it was an amazing place with many great people, including Arthur van Hoff who developed GoodNeWS/HyperNeWS/HyperLook! Unfortunately I never got the chance to meet Donald Michie (maybe he dropped by but I didn't recognize him), but I know from his reputation what a great guy he was. I did hear a funny story about him: Donald Michie once overheard his secretary telling someone on the phone how to pronounce his name (in a Scottish accent): "It's Donald, as in Duck, and Michie, as in Mouse." He was so pissed he refused to speak to her for a month! ;) For what it's worth, that's a great way to remember how his name is pronounced! When I was there in '92 it was being (mis)run by some upper crust Tory gentleman who specialized in bailing out failing companies. He was so uptight he got pissed off I'd written the address as "North Hangover St." on the white board! Sheez. Instead putting him in charge, they should have just called Old Glory: https://www.youtube.com/watch?v=KXnL7sdElno https://www.youtube.com/watch?v=KXnL7sdElno I've been writing an article (still in draft) about the work I did at the Turing Institute (HyperLook): https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-goodnews-99f411e58ce4 https://medium.com/@donhopkins/hyperlook-nee-hypernews-nee-g... Fun times! Anyway sorry to interrupt.
sgt101 8y ago
Formalisations do break down in the limit, Wittgenstein showed everyone that, but the denial is a deep and powerful choice "he must transcend these propositions and then he will see the world alright." There were engine room troubles for symbolic AI, committed choice meant that reasoners were unpredictable and impractical. We saw 100k improvement of compute change sub symbolic AI, and it has done the same for symbolic, and there have been some really smashing advances in the algorithms. Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what? Probabilistic formalisms and MCMC have changed what we can write down and reason about effectively as well. I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? I was more interested to read the HN reaction! My team has had lots of success creating models using our cluster (in 2014) and then a cluster of GPU's which we bought because we could see that this was the way to go. We haven't gone and spent 100x more since then, and yet we now can see plotted out that the leading teams are doing this. The hardware has not improved at a rate of doubling every 3.4mths, it's hardly improved at all for 12 mths. What's happening is that people are spending money to make waves. This is denial big time. Brute force is a wall, you can climb a few bricks, but the top is unobtainable. There have been some improvements in tree search and there have been some insights in terms of where we are wasting time with optimisations that aren't working but I've always thought that DL is brute force at heart and I think we are where we are with it now, and Alphazero is where we are going to be until someone really sees a new way.
- YeGoblynQueenne 8y ago
  >> I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? Aye. My reaction to that graph was that this rate of progress can't be sustainable. If industry was throwing good money after bad to find out for how long you can keep ussing bubblesort given more and more compute, before having to develop a better sorting algorithm- that's what it would look like. >> Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what? ASP is interesting. There are symbolic learning techniques that learn answer set programs from data, did you know that?
YeGoblynQueenne 8y ago
My apologieis but there is an ocean of misconceptions and a galaxy of misinformation, in that comment. >> The complexity of language looks finite. I have to ask what your defintion of "language" is because by formal language theory the only languages that are finite are, well, finite languages- which are simple than regular languages, which can be descirbed by a regular expression. And I'm pretty sure that nobody has ever managed to write a regular expression that can describe all of human language. >> Translation between all the European languages works fairly well. Asian languages, not so much yet. In fact, translation between "all the European lagnuages" does not work at all well. For limited domains and for languages with lots of examples of translation, it works alright. Stray from that assumption and the results quickly descent into incomprehensible gibberish. >> We're way ahead of where things were in the "AI winter", 1985-2005. This time the startups make money and do useful things. Making money is a measure of progress of the industry- not of the science. >> AI used to be tiny - about 20-50 people at MIT, CMU, and Stanford. That was true back in the '50s, when the field began. In the '80s and '90s, at the hight of GOFAI and expert systems, there were several thousands of researchers working on AI. For example, the 5th Generation Computer project was a massive effort by the Japanese Ministry of International Trade and Industry which encompassed pretty much all of that country's computer industry with huge loads of funding- not to mention the reaction in the West who panicked thinking that the Japanese were aobut to do to their computer industry what they had done to its car industry. You might have heard of the failure of the project- but it took thousands of people a long time to fail. So, no, AI was not tiny, by any sense of the word, for any time after the first years of its birth. >> (Me: MSCS, Stanford, 1985. I met most of the greats of classical logic-based AI. Trying to hammer the real world into predicate calculus just doesn't work. The expert systems guys were in denial big-time about this.) Last week I met a guy with a background in theoretical physics, who works in high-performance computing. Am I now qualified to contribute an opinion about theoretical physics and high-performance computing?

empath75 8y ago

If languages were infinitely complex, humans couldn’t do it either.