8 ms·

As someone who has been into AI since the mid-90s, I continue to be deeply disappointed with the overall rate of progress, this includes with all of the machine

As someone who has been into AI since the mid-90s, I continue to be deeply disappointed with the overall rate of progress, this includes with all of the machine-learning and stats based recasting of what's meant by AI. The big fundamental problems are all the same, and we don't appear to have much progress on them. We have figured out very clever ways of teaching extremely useful new tricks to computers. I don't mean to denigrate the value of that work, it just resembles regular programming, but by different means. Whatever your definition of intelligence, there's little to no generalizability of what's happening right now.

markan 9y ago
Totally agree with you. You might be interested in basicai.org. We're trying to address those big fundamental problems you reference.
indubitable 9y ago
A lot here depends on how you view generalization. Consider the Atari playing AI developed by Deepmind. It reached superhuman capability on a variety of different games given no domain specific knowledge. It had access to just the visual information and its score. It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score. But I think you have to consider that life itself starts in the exact same way. We're little more than a vast number of evolutions starting from very simple organisms who's only purpose was to seek out sustenance. They had an extremely simplified domain with limited input thanks to fewer senses. And they also received a score. They were 'informed' of their sustenance and if it fell below a certain level, that was game over which they were also informed of --- at least in some manner of speaking. I think the ultimate issue is that it's rather disappointing how unexciting it all is. And so we constantly shift the goal posts. Going back not that many decades ago it was believed that a computer being able to defeat humans at chess would signal genuine intelligence. Of course we managed to achieve that, but the matter of how it was done being so unexciting and uninteresting led us to shift the goal posts. The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined. Nonetheless we still now go, "Nah.. that's not REAL intelligence either." But I think we will, up to the day we do genuinely create generalized intelligence argue that it's not 'REAL' intelligence. And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards. Because it won't be magical, or exciting. I don't think there's ever going to be a "we've truly done it" moment. It's just going to be clever ways of teaching increasingly sophisticated tricks leading up to the point that a creation becomes more effective at most of any task, including assimilating and producing new knowledge and expertise, than we are.
- Cacti 9y ago
  Do we even have a good definition of what it means to generalize? I keep seeing this term thrown about and while it makes intuitive sense, and certainly train/test/validation splits are useful, I sort of question the whole premise. I mean, when we consider all possible sets, all optimization problems perform equally well (or poorly), so in the most general sense, generalization is impossible. Which means we have to restrict our sets somehow, but to do that we have to have some measurement as to how they relate, but how do you know in advance? In a sense what the machine is learning is a distance metric between these sets, but the only way to know it's working is to actually run it. To say in advance "well, this machine generalizes this well on this class of problems" seems like an awful stretch. So much of what we mean when we say "this model generalizes well" seems like baked-in human assumptions about the data distributions that may not really have any basis in reality.
  - moxious 9y ago
    Generalize does need to be more specific in order to make sense. In current ML work, generalizability means something else than what it typically used to mean in AI work. One way to look at what generalizability is would be like this: problem domains typically come with sets of axioms. Generalizability is the ability for a solution approach to work across different domains that don't have 100% axiom overlap. The wider the difference between axiom sets for a domain, the more difficult / impressive the generalization. Solving two problems within the same domain, necessarily sharing a 100% overlapping axiom set, is not generalization at all. The reason the axioms supporting the domain matter is that they (in part) guide which heuristics work, and how they should be applied. And that's the sort of generalizability that is missing from current work: some solutions can make some pre-programmed choices about applying different heuristics depending on the problem set (driverless cars are doing this now). This is the "bag of tricks" approach. But they don't typically morph in how they are applied, or the end that they're trying to accomplish when they're applied.
    - KGIII 9y ago
      I have pondered this and have decided what my standards are. I realize I don't have the authority to set those standards for everyone. GAI is, to me, when machine is able to be given a problem and then, without prompting, decides which data to consume to learn how to solve that problem. It could be told to optimize an automotive design for a 5% efficiency increase without a loss of safety features and while keeping the performance the same - and then go out and figure out what data it needs to learn so that it can solve that task. It would assemble and process that data and then come up with the answer, which might just be that it is impossible with current tech and here is what is needed and this is how to do it. That's rather verbose and I'm absolutely not the person who gets to define it. But, when someone says AI, that is how I think of it. More so when they say general AI.
SomeStupidPoint 9y ago
Could you outline some of the problems you think current research isn't addressing?
- deleted 9y ago
  [deleted]
- markan 9y ago
  Not the GP, but some examples are: (1) how do you get a computer to have a human-like train of thought? (2) how do you get a computer to acquire new concepts (e.g. "debt", "global warming", "weed") and then reason about them correctly, without any reprogramming? (3) automated acquisition of common sense through experience (e.g. "if you pour water on the floor you will get a puddle") (4) deep natural language understanding (i.e. how do you make a chatbot that really understands, and isn't just a thin illusion of understanding).
  - cassowary 9y ago
    (4) is an interesting question. Unfortunately it's much harder to understand than it is to ask. For instance, to people really understand, rather than just providing a thin illusion of understanding? What does it actually mean to understand something? Can you make a test that can distinguish arbitrary systems which truly understand from those which provide a thin illusion of understanding?
    - averagewall 9y ago
      This is a real problem for physics teachers - you want to find out if the students understand a concept: Ask them to state it - they memorize the text book definition. Ask them to apply it to a specified problem - they scan the problem for values of variables and look up a formula list to find one that has those variables. Ask them to explain why their answer is correct and they form a grammatically correct explanation made by plucking phrases from the problem description and linking them to the answer with "so" or "because". It feels like they don't understand but they actually can get a long way (ie. not fail) like that - it's certainly human level understanding, even if it's not what the smartest of us are capable of. Personally, I think understanding is a continuum from special case memorization at the bottom, up to being able to link with a lot of other concepts at the top. There's no bright line between "truly understands" and "illusion".
darawk 9y ago
I think part of the problem is that humans like to define their own intelligence in grandiose terms. Prior to their being solved, object identification, human-level speech recognition, handwriting recognition, machine translation and many other tasks were thought to require general intelligence. But once we got the machines doing them for us, we decided they weren't so hard after all. From this you can conclude one of two things: 1. We're on the wrong track, and don't understand our own intellects at all. 2. Human intelligence is just a collection of these same hacks, possibly ensembled by some relatively thin meta-algorithm.
- moxious 9y ago
  #2 is definitely wrong. We aren't a collection of similar hacks; we are qualitatively different because we include the ability to gain new heuristics and to cross apply the ones we already have. No clever library of strung together hacks will have these 2 properties. In the same way, an ant hill is not just a bunch of ants, it has so many emergent properties that thinking of it in terms of its components is a mistake We should however think more expansively about intelligence than just the sort humans have. The goal doesn't need to be straight mimicry of humans
  - darawk 9y ago
    A neural network that can train neural networks is capable of learning new heuristics. That is a thin meta-algorithm on top of a standard function approximation algorithm. EDIT: I should also add that reinforcement learning more directly falsifies your claim.
  - marcosdumay 9y ago
    > In the same way, an ant hill is not just a bunch of ants, it has so many emergent properties that thinking of it in terms of its components is a mistake The thing about emergent behavior is that you just have to take enough entities and organize them the correct way, and the behavior appears, unexpectedly, and out of nowhere. If it is really emergent, then nobody (what includes me and you) has no idea how far we are from a general intelligence at all.
  - 9y ago
deleted 9y ago
[deleted]
ghaff 9y ago
The state of things seems to be that we've made really good progress with deep learning/neural networks/MLPs with the result that there's impressive fuzzy pattern recognition based on supervised learning. Which is great and has a lot of practical value but it's just one set of related techniques. Cognitive science, on the other hand, has arguably not made a lot of great progress. We don't really understand how we learn. We don't really understand how language works. Etc. But arguably a lot of this sort of thing needs to be part of "AI" to get beyond problems that are amenable to pattern recognition.
log_base_login 9y ago
Most of what your brain does is statistical reasoning. Much of the rest is logical inference. You choose words in the order you choose them because as a child, your mind learned that, statistically speaking, certain words had certain meaning, and when used would engender certain results. All intelligent decisions follow that pattern. Training, comparison to expectations, memorize the difference, adjust behavior, rinse and repeat. Sound familiar? Most of the intelligent mechanisms within the mind are employed through similar statistical inference, and when it's not done statistically (say, when you are inspired by some flash of 'insight' or when you dream things that have little obvious statistical grounding) there is still some reasoning within the mind that is allowing room for stochasticity to infer intelligence that is statistically calculated to be beneficial, when all other reasoning has failed. Tangentially, it might even be true that the irrational mind is simply one that uses stochastic reasoning more often than is deemed sane by our social norms. Don't overstate intelligence like it's some amazing technology that we can't understand. Consciousness is more difficult to parlay into a black box model, but intelligence, reasoning, and 'wise' decision making is very much rooted in making statistically beneficial choices and deriving logic from the order we recognize via inductive and deductive reasoning. If it sounds like I am oversimplifying, please understand that I hold the brain and the amazing parallel processing power it wields in high regard, and it's no mean feat for someone like Einstein, to use an obvious example of great intelligence, to intuit relativistic frameworks using only thought experiments, but his understanding (and everyone else's, for that matter) were built on many, many layers of statistical reasoning.
Animats 9y ago
As someone who has been into AI since the late-70s, I am impressed with the overall rate of progress in the last 10 years. The whole field was stuck from the mid-1980s to about 2000. This is called the "AI Winter", and it really sucked. AI used to be tiny - 20-50 people at CMU, Stanford, and MIT, and a few smaller groups elsewhere. All the '80s startups went bust. Now there are real applications that work and are widely deployed. Many classic problems, such as reading text and handwriting, have been solved and the solutions widely deployed commercially. Speech recognition is getting good and is deployed widely. Face recognition works. Automatic driving is working experimentally. There are now hundreds of thousands of people doing AI-type research. Each new idea in AI has had a ceiling. Machine learning has a ceiling, but it's one high enough that the technology is good for something and generates enough revenue to finance its own R&D. Machine learning is basically a form of high-dimensional multivariate optimization in possibly bumpy spaces. That's a hard problem, but considerable progress has been made, and the massive compute power necessary is now available. This is great. It's not everything, but it's real progress. We're going to need another big idea to get beyond the ceiling of machine learning. No idea what that idea is.
- freyir 9y ago
  I’m an outsider, but what’s the recent “big idea”? It seems to be throwing more compute power and larger training sets at essentially an old technique. This has led to a big improvement in performance, but I don’t see the big conceptual breakthrough.
  - sgt101 9y ago
    I think that Bayesian Program Learning as pioneered by Lake et-al is a big idea. I think that GAN's probably qualify - although you can see that emerging in the SAB series of conferences in the 90's if you read the papers. On the other hand I do see a lot of small innovations that are enabling many people to create incremental improvements and applications. I feel that that the exploration of the field has been very weak and our overall knowledge is limited and not widely shared. Perhaps the improvements like MCMC search for bayesian reasoning, causality, counterfactuals, GPU's and TPU's and FPGA's and the access to very large data sets for training, forward training and so on will be the actual breakthrough.

markan 9y ago

Totally agree with you. You might be interested in basicai.org. We're trying to address those big fundamental problems you reference.

indubitable 9y ago

A lot here depends on how you view generalization. Consider the Atari playing AI developed by Deepmind. It reached superhuman capability on a variety of different games given no domain specific knowledge. It had access to just the visual information and its score. It's natural to claim that's not generalizing since it's in such a specific domain as well as the fact that it had access to its score. But I think you have to consider that life itself starts in the exact same way. We're little more than a vast number of evolutions starting from very simple organisms who's only purpose was to seek out sustenance. They had an extremely simplified domain with limited input thanks to fewer senses. And they also received a score. They were 'informed' of their sustenance and if it fell below a certain level, that was game over which they were also informed of --- at least in some manner of speaking. I think the ultimate issue is that it's rather disappointing how unexciting it all is. And so we constantly shift the goal posts. Going back not that many decades ago it was believed that a computer being able to defeat humans at chess would signal genuine intelligence. Of course we managed to achieve that, but the matter of how it was done being so unexciting and uninteresting led us to shift the goal posts. The achievements we're regularly producing now a days dwarf anything those speaking of chess=intelligence times could have even imagined. Nonetheless we still now go, "Nah.. that's not REAL intelligence either." But I think we will, up to the day we do genuinely create generalized intelligence argue that it's not 'REAL' intelligence. And I imagine we'll be arguing that's not 'REAL' intelligence even afterwards. Because it won't be magical, or exciting. I don't think there's ever going to be a "we've truly done it" moment. It's just going to be clever ways of teaching increasingly sophisticated tricks leading up to the point that a creation becomes more effective at most of any task, including assimilating and producing new knowledge and expertise, than we are.

Cacti 9y ago
Do we even have a good definition of what it means to generalize? I keep seeing this term thrown about and while it makes intuitive sense, and certainly train/test/validation splits are useful, I sort of question the whole premise. I mean, when we consider all possible sets, all optimization problems perform equally well (or poorly), so in the most general sense, generalization is impossible. Which means we have to restrict our sets somehow, but to do that we have to have some measurement as to how they relate, but how do you know in advance? In a sense what the machine is learning is a distance metric between these sets, but the only way to know it's working is to actually run it. To say in advance "well, this machine generalizes this well on this class of problems" seems like an awful stretch. So much of what we mean when we say "this model generalizes well" seems like baked-in human assumptions about the data distributions that may not really have any basis in reality.
- moxious 9y ago
  Generalize does need to be more specific in order to make sense. In current ML work, generalizability means something else than what it typically used to mean in AI work. One way to look at what generalizability is would be like this: problem domains typically come with sets of axioms. Generalizability is the ability for a solution approach to work across different domains that don't have 100% axiom overlap. The wider the difference between axiom sets for a domain, the more difficult / impressive the generalization. Solving two problems within the same domain, necessarily sharing a 100% overlapping axiom set, is not generalization at all. The reason the axioms supporting the domain matter is that they (in part) guide which heuristics work, and how they should be applied. And that's the sort of generalizability that is missing from current work: some solutions can make some pre-programmed choices about applying different heuristics depending on the problem set (driverless cars are doing this now). This is the "bag of tricks" approach. But they don't typically morph in how they are applied, or the end that they're trying to accomplish when they're applied.
  - KGIII 9y ago
    I have pondered this and have decided what my standards are. I realize I don't have the authority to set those standards for everyone. GAI is, to me, when machine is able to be given a problem and then, without prompting, decides which data to consume to learn how to solve that problem. It could be told to optimize an automotive design for a 5% efficiency increase without a loss of safety features and while keeping the performance the same - and then go out and figure out what data it needs to learn so that it can solve that task. It would assemble and process that data and then come up with the answer, which might just be that it is impossible with current tech and here is what is needed and this is how to do it. That's rather verbose and I'm absolutely not the person who gets to define it. But, when someone says AI, that is how I think of it. More so when they say general AI.

SomeStupidPoint 9y ago

Could you outline some of the problems you think current research isn't addressing?

deleted 9y ago
[deleted]
markan 9y ago
Not the GP, but some examples are: (1) how do you get a computer to have a human-like train of thought? (2) how do you get a computer to acquire new concepts (e.g. "debt", "global warming", "weed") and then reason about them correctly, without any reprogramming? (3) automated acquisition of common sense through experience (e.g. "if you pour water on the floor you will get a puddle") (4) deep natural language understanding (i.e. how do you make a chatbot that really understands, and isn't just a thin illusion of understanding).
- cassowary 9y ago
  (4) is an interesting question. Unfortunately it's much harder to understand than it is to ask. For instance, to people really understand, rather than just providing a thin illusion of understanding? What does it actually mean to understand something? Can you make a test that can distinguish arbitrary systems which truly understand from those which provide a thin illusion of understanding?
  - averagewall 9y ago
    This is a real problem for physics teachers - you want to find out if the students understand a concept: Ask them to state it - they memorize the text book definition. Ask them to apply it to a specified problem - they scan the problem for values of variables and look up a formula list to find one that has those variables. Ask them to explain why their answer is correct and they form a grammatically correct explanation made by plucking phrases from the problem description and linking them to the answer with "so" or "because". It feels like they don't understand but they actually can get a long way (ie. not fail) like that - it's certainly human level understanding, even if it's not what the smartest of us are capable of. Personally, I think understanding is a continuum from special case memorization at the bottom, up to being able to link with a lot of other concepts at the top. There's no bright line between "truly understands" and "illusion".

darawk 9y ago

I think part of the problem is that humans like to define their own intelligence in grandiose terms. Prior to their being solved, object identification, human-level speech recognition, handwriting recognition, machine translation and many other tasks were thought to require general intelligence. But once we got the machines doing them for us, we decided they weren't so hard after all. From this you can conclude one of two things: 1. We're on the wrong track, and don't understand our own intellects at all. 2. Human intelligence is just a collection of these same hacks, possibly ensembled by some relatively thin meta-algorithm.

moxious 9y ago
#2 is definitely wrong. We aren't a collection of similar hacks; we are qualitatively different because we include the ability to gain new heuristics and to cross apply the ones we already have. No clever library of strung together hacks will have these 2 properties. In the same way, an ant hill is not just a bunch of ants, it has so many emergent properties that thinking of it in terms of its components is a mistake We should however think more expansively about intelligence than just the sort humans have. The goal doesn't need to be straight mimicry of humans
- darawk 9y ago
  A neural network that can train neural networks is capable of learning new heuristics. That is a thin meta-algorithm on top of a standard function approximation algorithm. EDIT: I should also add that reinforcement learning more directly falsifies your claim.
- marcosdumay 9y ago
  > In the same way, an ant hill is not just a bunch of ants, it has so many emergent properties that thinking of it in terms of its components is a mistake The thing about emergent behavior is that you just have to take enough entities and organize them the correct way, and the behavior appears, unexpectedly, and out of nowhere. If it is really emergent, then nobody (what includes me and you) has no idea how far we are from a general intelligence at all.
- 9y ago

deleted 9y ago

[deleted]

ghaff 9y ago

The state of things seems to be that we've made really good progress with deep learning/neural networks/MLPs with the result that there's impressive fuzzy pattern recognition based on supervised learning. Which is great and has a lot of practical value but it's just one set of related techniques. Cognitive science, on the other hand, has arguably not made a lot of great progress. We don't really understand how we learn. We don't really understand how language works. Etc. But arguably a lot of this sort of thing needs to be part of "AI" to get beyond problems that are amenable to pattern recognition.

log_base_login 9y ago

Most of what your brain does is statistical reasoning. Much of the rest is logical inference. You choose words in the order you choose them because as a child, your mind learned that, statistically speaking, certain words had certain meaning, and when used would engender certain results. All intelligent decisions follow that pattern. Training, comparison to expectations, memorize the difference, adjust behavior, rinse and repeat. Sound familiar? Most of the intelligent mechanisms within the mind are employed through similar statistical inference, and when it's not done statistically (say, when you are inspired by some flash of 'insight' or when you dream things that have little obvious statistical grounding) there is still some reasoning within the mind that is allowing room for stochasticity to infer intelligence that is statistically calculated to be beneficial, when all other reasoning has failed. Tangentially, it might even be true that the irrational mind is simply one that uses stochastic reasoning more often than is deemed sane by our social norms. Don't overstate intelligence like it's some amazing technology that we can't understand. Consciousness is more difficult to parlay into a black box model, but intelligence, reasoning, and 'wise' decision making is very much rooted in making statistically beneficial choices and deriving logic from the order we recognize via inductive and deductive reasoning. If it sounds like I am oversimplifying, please understand that I hold the brain and the amazing parallel processing power it wields in high regard, and it's no mean feat for someone like Einstein, to use an obvious example of great intelligence, to intuit relativistic frameworks using only thought experiments, but his understanding (and everyone else's, for that matter) were built on many, many layers of statistical reasoning.

Animats 9y ago

As someone who has been into AI since the late-70s, I am impressed with the overall rate of progress in the last 10 years. The whole field was stuck from the mid-1980s to about 2000. This is called the "AI Winter", and it really sucked. AI used to be tiny - 20-50 people at CMU, Stanford, and MIT, and a few smaller groups elsewhere. All the '80s startups went bust. Now there are real applications that work and are widely deployed. Many classic problems, such as reading text and handwriting, have been solved and the solutions widely deployed commercially. Speech recognition is getting good and is deployed widely. Face recognition works. Automatic driving is working experimentally. There are now hundreds of thousands of people doing AI-type research. Each new idea in AI has had a ceiling. Machine learning has a ceiling, but it's one high enough that the technology is good for something and generates enough revenue to finance its own R&D. Machine learning is basically a form of high-dimensional multivariate optimization in possibly bumpy spaces. That's a hard problem, but considerable progress has been made, and the massive compute power necessary is now available. This is great. It's not everything, but it's real progress. We're going to need another big idea to get beyond the ceiling of machine learning. No idea what that idea is.

freyir 9y ago
I’m an outsider, but what’s the recent “big idea”? It seems to be throwing more compute power and larger training sets at essentially an old technique. This has led to a big improvement in performance, but I don’t see the big conceptual breakthrough.
- sgt101 9y ago
  I think that Bayesian Program Learning as pioneered by Lake et-al is a big idea. I think that GAN's probably qualify - although you can see that emerging in the SAB series of conferences in the 90's if you read the papers. On the other hand I do see a lot of small innovations that are enabling many people to create incremental improvements and applications. I feel that that the exploration of the field has been very weak and our overall knowledge is limited and not widely shared. Perhaps the improvements like MCMC search for bayesian reasoning, causality, counterfactuals, GPU's and TPU's and FPGA's and the access to very large data sets for training, forward training and so on will be the actual breakthrough.