3 ms·

"I want to switch gears slightly and talk about other ways we use humans in modern research." I thought the first argument, regarding the use of humans to cons

by jonmc12 17y ago

"I want to switch gears slightly and talk about other ways we use humans in modern research." I thought the first argument, regarding the use of humans to conserve resources associated with time, data and domain expertise, was right on and very articulate. I expected this argument to be extended in the '40 years' section of the article. After all, isn't the shift to computer-driven learning simply a threshold between te utility / cost of supervised/semi-supervised learning vs unsupervised learning? In other words, couldn't you say that at some point the 'ways we use humans' becomes too expensive vs non-human methods? When this threshold is crossed is when 'non-biological entities have more reasoning and processing power than biological entities' Maybe just another way of saying the same thing. In any case, a great post. Worth reading closely in my opinion.

sdrinf 17y ago
Um, no. The trade-off isn't between utilizing humans VS utilizing computers. It's the R&D cost of building simple, and proven systems, and paying peanuts into mturk VS the R&D of complex, and sophisticated systems * probability of working. In other words, our tightest bottleneck is not the cost of supervising, but rather our understanding of the strategies, and abilities to tackle a given problem domain using only the limited tools of a fancy abacus. And since this is HN, I'd like to digress here, and propose the curse of researching such strategies, and abilities, is that from an investor's POV, the math simply doesn't add up. The R&D phase can go on practically as long as the budget; probability of even incremental success is slim; and except for a couple of well-defined domains, applicability is negligible. Investors have 2 AI-winters[1] to point, and laugh at, which I suspect is the main reason behind ML/IR/NLP to re-market their terminology. This is a very deep problem I would pay for humans to tackle. To deconstruct the second part of your argument, the notion of "having more reasoning and processing power than biological entities" is naive at best. In the context of human brains neither "reasoning power", nor "processing power" could be quantified in any way that would even resemble our quantification of these properties in computers. It's like comparing the Na'vi[2] to Steve Jobs. This naivety stems from the common misconception of presuming a singular/general reasoning capability; instead of a fairly large amount of highly specialized modules, each responsible to a single type of cognition, and reasoning. An important point to note here, is that what our brain does VS what we are building[3] differ significantly: the market is much more interested in functional diversification, than exchanging pennies of supervising for pennies of computing power. In conclusion, the "reasoning, and processing power" of humans, and ML/IR/NLP/etc systems are not only incomparable in terms of quantification, but also in their evolutionary environment, and thus their functional goals. [1] http://en.wikipedia.org/wiki/Ai_winter http://en.wikipedia.org/wiki/Ai_winter [2] http://en.wikipedia.org/wiki/Navi#Na.27vi http://en.wikipedia.org/wiki/Navi#Na.27vi [3] http://stackoverflow.com/questions/1050696/the-business-of-artificial-intelligence http://stackoverflow.com/questions/1050696/the-business-of-a...
- jonmc12 17y ago
  The article was addressing the validity of the viewpoint that 'non-biological entities have more reasoning and processing power than biological entities...I think this is all going to happen in the next 40 years...' I tend to agree with you that there is no known quantitative measure to answer this question. Instead (because there is no known quantification method), I proposed the question - if a given learning problem can be solved most cost effectively using only computers, has it satisfied this requirement (of a non-biological entity achieving superior reasoning on a specific problem(s))? You said 'no', due to the cost of R&D. However, computer-driver R&D, particularly in the form of generative hypothesis testing may one day be able to perform the job of 'understanding the strategies' more accurately and at a lower cost than a human. Lets say your cost of learning is a function of the resource limitations the article laid out. ie, Cost of Learning = Cost of R&D + Cost of Data + Cost of Domain expertise. What I am suggesting is that if this Cost of Learning is lower for a computer alone than it is for a computer + human, than this is perhaps an adequate test to say 'In this problem area, a non-biological entity has achieved superior reasoning compared to a biological entity'. btw, I tend to agree with your digression as well. This is a deep problem that is not really investor friendly, but needs sponsorship one way or another to advance the state of the art.
  - sdrinf 17y ago
    Picking single-aspect reasoning lightens the condition (and thus, the strength of the argument) quite a bit. If you take a single learning problem domain, there are a number of fully unsupervised ML/IR systems already out there with superhuman reasoning capabilities. The second part of your argument is a funky one. Fully autonomous R&D-capable AIs (AKA self-improving that is, a problem-solving / reasoning system, that is capable of creating new, and improving upon existing problem-solving / reasoning systems -which, coincidently, might include itself) are the modern equalent of the philosopher's stone, and has not only withstanded attacks in ways that few other problem domains did, but seems to be an AI-hard[1] problem by itself. As I laid down above, market forces does not favour these kind of R&D; thus I predict this problem to be unsolved for the short-term future. On the other paw, the second parameter of your cost function - "cost of data" - can be exchanged by "amount of available data" -which seems to be in an explosive boom from where I stand. Thus, while we're talking predictions, I would hypothise a short-term future with a slow, and incremental improvements upon tackling domains combined with an exponentially growing amount of available data, making even relatively naive learning methods to have results well within the optimal range. It will still require humans. It will still require R&D. But the "cost of learning" will be so much lower, that the process will be routinely employed (successfully) by developers; thus diversifying the overall software market even further. I know it doesn't make for a fancy movie, but throwing away 6.5M years of R&D by mother evolution doesn't seems to be a wise diversification move. [1] http://en.wikipedia.org/wiki/AI-complete http://en.wikipedia.org/wiki/AI-complete

by jonmc12 17y ago

sdrinf 17y ago

Um, no. The trade-off isn't between utilizing humans VS utilizing computers. It's the R&D cost of building simple, and proven systems, and paying peanuts into mturk VS the R&D of complex, and sophisticated systems * probability of working. In other words, our tightest bottleneck is not the cost of supervising, but rather our understanding of the strategies, and abilities to tackle a given problem domain using only the limited tools of a fancy abacus. And since this is HN, I'd like to digress here, and propose the curse of researching such strategies, and abilities, is that from an investor's POV, the math simply doesn't add up. The R&D phase can go on practically as long as the budget; probability of even incremental success is slim; and except for a couple of well-defined domains, applicability is negligible. Investors have 2 AI-winters[1] to point, and laugh at, which I suspect is the main reason behind ML/IR/NLP to re-market their terminology. This is a very deep problem I would pay for humans to tackle. To deconstruct the second part of your argument, the notion of "having more reasoning and processing power than biological entities" is naive at best. In the context of human brains neither "reasoning power", nor "processing power" could be quantified in any way that would even resemble our quantification of these properties in computers. It's like comparing the Na'vi[2] to Steve Jobs. This naivety stems from the common misconception of presuming a singular/general reasoning capability; instead of a fairly large amount of highly specialized modules, each responsible to a single type of cognition, and reasoning. An important point to note here, is that what our brain does VS what we are building[3] differ significantly: the market is much more interested in functional diversification, than exchanging pennies of supervising for pennies of computing power. In conclusion, the "reasoning, and processing power" of humans, and ML/IR/NLP/etc systems are not only incomparable in terms of quantification, but also in their evolutionary environment, and thus their functional goals. [1] http://en.wikipedia.org/wiki/Ai_winter http://en.wikipedia.org/wiki/Ai_winter [2] http://en.wikipedia.org/wiki/Navi#Na.27vi http://en.wikipedia.org/wiki/Navi#Na.27vi [3] http://stackoverflow.com/questions/1050696/the-business-of-artificial-intelligence http://stackoverflow.com/questions/1050696/the-business-of-a...

jonmc12 17y ago
The article was addressing the validity of the viewpoint that 'non-biological entities have more reasoning and processing power than biological entities...I think this is all going to happen in the next 40 years...' I tend to agree with you that there is no known quantitative measure to answer this question. Instead (because there is no known quantification method), I proposed the question - if a given learning problem can be solved most cost effectively using only computers, has it satisfied this requirement (of a non-biological entity achieving superior reasoning on a specific problem(s))? You said 'no', due to the cost of R&D. However, computer-driver R&D, particularly in the form of generative hypothesis testing may one day be able to perform the job of 'understanding the strategies' more accurately and at a lower cost than a human. Lets say your cost of learning is a function of the resource limitations the article laid out. ie, Cost of Learning = Cost of R&D + Cost of Data + Cost of Domain expertise. What I am suggesting is that if this Cost of Learning is lower for a computer alone than it is for a computer + human, than this is perhaps an adequate test to say 'In this problem area, a non-biological entity has achieved superior reasoning compared to a biological entity'. btw, I tend to agree with your digression as well. This is a deep problem that is not really investor friendly, but needs sponsorship one way or another to advance the state of the art.
- sdrinf 17y ago
  Picking single-aspect reasoning lightens the condition (and thus, the strength of the argument) quite a bit. If you take a single learning problem domain, there are a number of fully unsupervised ML/IR systems already out there with superhuman reasoning capabilities. The second part of your argument is a funky one. Fully autonomous R&D-capable AIs (AKA self-improving that is, a problem-solving / reasoning system, that is capable of creating new, and improving upon existing problem-solving / reasoning systems -which, coincidently, might include itself) are the modern equalent of the philosopher's stone, and has not only withstanded attacks in ways that few other problem domains did, but seems to be an AI-hard[1] problem by itself. As I laid down above, market forces does not favour these kind of R&D; thus I predict this problem to be unsolved for the short-term future. On the other paw, the second parameter of your cost function - "cost of data" - can be exchanged by "amount of available data" -which seems to be in an explosive boom from where I stand. Thus, while we're talking predictions, I would hypothise a short-term future with a slow, and incremental improvements upon tackling domains combined with an exponentially growing amount of available data, making even relatively naive learning methods to have results well within the optimal range. It will still require humans. It will still require R&D. But the "cost of learning" will be so much lower, that the process will be routinely employed (successfully) by developers; thus diversifying the overall software market even further. I know it doesn't make for a fancy movie, but throwing away 6.5M years of R&D by mother evolution doesn't seems to be a wise diversification move. [1] http://en.wikipedia.org/wiki/AI-complete http://en.wikipedia.org/wiki/AI-complete