11 ms·

Is there a simple algorithm for intelligence?

129 points by rcoppolo 11y ago

giardini 11y ago
tl;dr - "We don't know[but please read my blog]."
eli_gottlieb 11y ago
>So 125 million base pairs is equivalent to 250 million bits of information. That's the genetic difference between humans and chimps! I don't mean to nitpick, but epigenetics is a thing. Also... how up-to-date on the research literature is this essay supposed to be?
- TheOtherHobbes 11y ago
  The information isn't just in the base pairs. It's in the entire evolved ecology (with optional industrial add-on pack) that makes the base pairs do something useful. How much of that could you take away and still get a human brain?
  - e_modad 11y ago
    Craig Venter says here[1] that Franz Och (of Google Translate fame) has done some analysis for his new company Human Longevity Inc that shows that about 50% of the genome is responsible for the brain. So the number is quite big and much larger than I would have thought. Excited to see more results from HLI. Comment at 13:12 [1] https://youtu.be/D3nIIKwwiLc?t=11m43s https://youtu.be/D3nIIKwwiLc?t=11m43s
    - bohm 11y ago
      Which is interesting considering we share about 85% of our genome with Zebra Fish.
      - e_modad 11y ago
        Source? That sounds a little high.
        bohm 11y ago
        From memory, but maybe a slight overstatement: http://www.sci-news.com/genetics/article01036.html http://www.sci-news.com/genetics/article01036.html 70 per cent of protein-coding human genes are related to genes found in the zebrafish (Danio rerio), and 84 per cent of genes known to be associated with human disease have a zebrafish counterpart.
cousin_it 11y ago
Given unlimited computing power, AIXItl [1] is a pretty simple algorithm that can provably solve all problems, in some sense, at least as well as any other algorithm. The idea is to simply dovetail over all possible algorithms and select the ones that fit observations best. That includes humans, if you believe as I do that humans are computable. With limited computing power, it's likely that the best algorithms for many different problems (possibly including "intelligence" however you define it) won't be simple, just like the best known algorithms for integer multiplication [2] aren't simple. In particular, AIXI variants will be hard to approximate, precisely because they dovetail over all possible algorithms. That said, it's very likely that the best algorithms for intelligence will be simpler and faster than humans, because humans are the stupidest possible creatures that can build a civilization (otherwise it would've happened earlier in our evolution). [1] http://www.hutter1.net/ai/paixi.htm http://www.hutter1.net/ai/paixi.htm [2] https://en.wikipedia.org/wiki/F%C3%BCrer%27s_algorithm https://en.wikipedia.org/wiki/F%C3%BCrer%27s_algorithm
- eli_gottlieb 11y ago
  >Given unlimited computing power, AIXItl [1] is a pretty simple algorithm that can provably solve all problems at least as well, in some sense, as any other algorithm. Technically, AIXI_{tl} solves all problems at least as well as any other algorithm in a given order of asymptotic time and length, those being the t and l. The problem is that it has a "trivial additive constant" which will make it take longer than the lifetimes of many stars to achieve its much-vaunted "optimal" asymptotic performance. In short, AIXI is the most utterly brute-force kind of "intelligence" you could possibly build, whose notion of "scaled down" bounded-rational inference is still astronomically intractable. It basically just amounts to throwing computing power at the problem, more computing power than anyone actually has. Mind, Schmidhuber and Hutter still get massive props for managing to cut out the philoso-wank that usually accompanies discussions of cognition and instead saying, "Let's just pose a very general inference problem and write an algorithm that solves it."
  - cousin_it 11y ago
    Yeah, that's why I said "given unlimited computing power". But theoretical work sometimes has a way of becoming practical and scary. Check out the work of Joel Veness on Monte Carlo AIXI, which learned to play Pac-Man on a single desktop computer, presumably faster than "the lifetimes of many stars".
meeper16 11y ago
It has to do with pattern matching. In particular, comparing vectors with one another for similarity or dissimilarity.
- knodi123 11y ago
  define "vector" in this context
kazinator 11y ago
Yes, there is a simple algorithm for intelligence. It's just a few lines of code. However, it requires a terabyte-sized table of data which has yet to be filled in.
- knodi123 11y ago
  there's a simple algorithm for running the latest version of windows. It's only a few lines of code, and a hundred gigs of compressed C++.
bohm 11y ago
This ignores that genetic code is meaningless without the translation machinery (ribosomes and the entire physical world of physics and chemistry). Genetic code without the translation machinery and the physical world it is translated into is meaningless. A multi-cellular embodied intelligence can acquire information about the patterns of the physical world by growing in it - the code is compact because much of the necessary information can be acquired during embodied growth. A disembodied intelligence such as AIs can't rely on external storage and acquisition of information during a cellular growth stage, and thus must likely be considerably more complex from the get-go.
- rndn 11y ago
  It also ignores the possibilities (1), that cells and tissues are extremely complex (and thus require a lot of genetic code), while the overall algorithm the cells and tissues perform might still be very simple, and conversely (2), that chimpanzees might be already extremely intelligent and the difference is rather a matter of embodiment (in particular the ability to produce language), and perhaps minor tweaks to neurons, to the cortex and/or other brain regions.
  - bohm 11y ago
    According to the Santiago Theory of Cognition "Living systems are cognitive systems, and living as a process is a process of cognition. This statement is valid for all organisms, with or without a nervous system." https://en.wikipedia.org/wiki/Santiago_Theory_of_Cognition https://en.wikipedia.org/wiki/Santiago_Theory_of_Cognition If this were a correct interpretation, even Bacteria could be seen as already incredibly intelligent. Going Zero to One (dead matter -> prokaryotes) is a significantly bigger step than going One to Five (prokaryotes -> humans).
    - Retric 11y ago
      Careful, modern prokaryotes also have billions of years of evolution behind them. The first organism where likely inefficient and would not last in our current hyper competitive ecosystems. Arguably, the first organism where really more just free floating proteins without even a cell wall.
ThomPete 11y ago
Wouldn't there have to be? Given that we evolved from much simpler organisms.
UhUhUhUh 11y ago
I hold the same belief. I also think that we don't pay enough attention to the form of processes whereas nature seems to place lots of emphasis on it. In fact, the more it goes, the more a form factor appears to be somehow determinant. Molecular biology with enzymes, genetics with histones, neurobiology with firing patterns, etc. We can see that now because we have access to finer grained observation. Form could be a higher-order type of content and could also be approximated with an algorithm. This all feeds in a previous discussion about intuition. Top-down or bottom-up?
300bps 11y ago
In the 16th century only a foolish optimist could have imagined that all these objects' motions could be explained by a simple set of principles. But in the 17th century Newton formulated his theory of universal gravitation, which not only explained all these motions, but also explained terrestrial phenomena such as the tides and the behaviour of Earth-bound projecticles. It turns out that Newton's ideas on gravity are a mathematical approximation that at least ignored relativistic effects. See: http://physics.stackexchange.com/questions/52165/newtonian-gravity-vs-general-relativity-exactly-how-wrong-is-newton http://physics.stackexchange.com/questions/52165/newtonian-g... Newton's gravity was a theory about a single fundamental force - and it turned out to be not 100% accurate. Rather than be an example of how intelligence might be similarly possible to explain with a simple theory, I think it offers a perfect counter-example of why it almost certainly cannot.
- habitue 11y ago
  Comets, and the planets and the tides, and apples falling from trees were found to be caused by the same underlying phenomena, despite seeming very different. Newton's insight that they were the same was not overturned by relativity, so the point stands quite well.
  - 300bps 11y ago
    Newton's insight that they were the same was not overturned by relativity I almost feel like you're replying to someone else because I never said what you are contradicting. I merely said that Newton's gravity is a simplification that doesn't take all variables into account. So to point to Newton's gravity as an example of, "Sometimes simple things can be right!" actually makes it more of a counter-example.
    - habitue 11y ago
      I was pointing out that the complex explanation is that comets have their own driving force, and tides have a separate driving force, and planets have their own reason for moving as they do, etc. Compared to that kind of hodge-podge, relativity is still a really simple theory. While relativity increased the complexity of the explanation of gravity, it didn't bring it anywhere near the level of complexity that might have been assumed before Newton. The author's argument is that our intelligence is likely not solely possible by the interaction of a large number of separate principles, but rather by some simple principle that manifests itself in different ways. I think if we find some simple principle that explains intelligence, and then later have to make a correction to the math of that principle, the resulting explanation will still be simpler than the idea that intelligence is made up of many different principles all of which have to be present.
emergentcypher 11y ago
Something something law of headlines something answer is "no".
thomasfoster96 11y ago
I'd think a 'simple' algorithm for intelligence would be even more distant than artificial general intelligence itself.
- knodi123 11y ago
  Simplicity is always only a little ways off. If I sat you next to god and told you to pair program and write an OS equivalent to Windows XP, with you driving the keyboard- you'd be sitting there for quite a while. But if I told you guys to pair program an AGI, you could probably finish in a day or so. A simple algorithm is a matter of finding just one or two basic discoveries. It could happen tomorrow! It probably won't, but that's the nature of discovery.
  - thomasfoster96 11y ago
    Well, supposing I'm not programming with god, rather I'm with a bunch of programmer friends, I still doubt the first artificial general intelligence (assuming we happen to be the people who make it) will be a beautifully simple piece of software. It'll probably be horribly hacked together and hard to maintain, and only in time will a simpler way to do things emerge. For example, I'd reckon we'll find that a lot of things that today only make sense to solve with a neural net are actually perfectly representable using a simple(ish) algorithm, but it's unlikely we'll find that out until after we create an artificial general intelligence and know more about the problem of intelligence. In other words, we'll solve the problem, and then realise how much easier it could have been.
    - knodi123 11y ago
      https://xkcd.com/224/ https://xkcd.com/224/
paulojreis 11y ago
> "I believe it's not in serious doubt that an intelligent computer is possible" - Yes, of course it is possible, if we change the meaning of "intelligent" to fit what a computer does. :) Generally, I think the problem here - as in many other situations - starts with the lack of rigor that computer science has when "naming" fields or, most exactly, appropriating concepts from other fields. * Computer Vision: "vision" has a defined meaning, even "organic-related". Signal processing and a bunch of algorithms aren't exactly what vision is and what we define as "vision"; * Machine learning: does the computer really learns, as understood by the common definition of "learning"? * Context awareness: if we consider common definitions, neither a few sensor readings are "context", nor acting according to said readings is "awareness". This might sound like nitpicking, but I truly think it's a real problem. Talking about vision, learning or intelligence put's the stake at what people or the general scientific community commonly perceive as those concepts. What's most frustrating to me is the contrast. Computer science research is thorough; why are we not thorough on naming our fields of study? Why do we appropriate concepts by "analogy", knowingly that analogies aren't truly exact?
- habitue 11y ago
  > Yes, of course it is possible, if we change the meaning of "intelligent" to fit what a computer does. :) I don't think that's what he's doing. Otherwise he could just define intelligence to mean what computers do now. Clearly he has at least roughly the same idea of intelligence you or I do. > "I believe it's not in serious doubt that an intelligent computer is possible" Do you have an objection to this specific statement when taken to mean general human-like intelligence being exhibited by computers? I have not heard anyone defend the opposite viewpoint often, and I'm interested if there are any compelling arguments why it's not possible.
  - turkeysandwich 11y ago
    Any algorithms we use to simulate intelligence with machines are excessively wasteful. There's not enough energy to simulate human intelligence at the same speed that humans think, without just essentially producing biological computers. And that's more of bio-engineering than anything relating to computing.
mathgenius 11y ago
> I believe it's not in serious doubt that an intelligent computer is possible - although it may be extremely complicated, and perhaps far beyond current technology - and current naysayers will one day seem much like the vitalists. Physicists (and this guy is one of them) certainly do have a robust set of taboos against considering consciousness as a quantum phenomena (whatever that might mean.) It's unfortunate because there is a huge resurgence going on in the physics of quantum many-body entanglement, quantum computers, foundations of quantum, etc. Alot of this meandering was just sidelined in the 1940s when world affairs intervened. However, in light of these new discoveries it's time to revisit the old vitalist arguments. Imho.
- TTPrograms 11y ago
  If consciousness is a quantum phenomenon it has coherence scales orders of magnitude beyond what we can achieve in labs at ~1 Kelvin. That is, it's pretty unlikely that quantum effects are having macro scale effects on the brain.
  - mathgenius 11y ago
    Yes, this is certainly the prevailing opinion among practicing physicists. I would like to point out though, that this is a comparison between a few years of tinkering in the lab with billions of years of evolution.
    - TTPrograms 11y ago
      That's a good point. We should probably not trust this "lab tinkering" because it's so new - nature already figured everything out already. I'll go back to hunting and gathering now...
- habitue 11y ago
  Let's say, for the sake of argument, that intelligence requires quantum effects to work. I don't see how going back to vitalist arguments will aid us in any way. Researching the phenomena, following evidence, making inferences from that evidence... that's how progress on that subject would be made. Not by going back to arguments made by people with less information than we have now. Specifically, if we follow the evidence, and find that somehow, beyond all odds, the vitalists were correct (once you add "quantumness!" to the mix), then it's clearly only by lucky guess they were right. It would be something we could look back at and say "wow, that's an interesting coincidence" rather than "wow, those vitalists had it right all along, we should have paid closer attention".
amelius 11y ago
There probably is a very simple algorithm for intelligence. It is just way to expensive to run it. (The reason is very simple, namely that the rules for the universe are probably very simple.)
norea-armozel 11y ago
Just from skimming the article it seems to me the problem with AI and probably just the whole matter of what people define as intelligence is that it assumes that the universe itself is not intelligent. Maybe the universe itself is intelligent but that consciousness is some other phenomena that gives intelligence an individual characteristic? It's just a random thought, I don't really know which way to go with it.
- davesque 11y ago
  I can relate. While I consider myself an agnostic with a slight atheist streak, I still like to muse on these sorts of thoughts occasionally. The universe does seem somehow innately intelligent and alive. Everything is in motion. Everything appears to have some kind of structure. Gotta be careful not to fall back on the "G" word though (not that you did) :).
  - norea-armozel 11y ago
    Personally, I think there might be a Creator, but that doesn't seem to be relevant as to the problem of consciousness. To me, when I see that physical processes act in a manner that is complex then it indicates that what we're trying to call intelligence is really just form of stateful structure. And the rest of intelligence is something else entirely different (and probably unrelated).
  - javajosh 11y ago
    "The Universe" is huge and empty and completely hostile to life. Most matter is in stars which are not empty and completely hostile to life. (And which are huge compared to us, but which are like specks of dust compared to "The Universe"). Our planet is a speck of dust rotating at a fortunate distance around another speck of (shining) dust. By mass, the majority of the planet is useless to life. Everything alive on Earth exists in a very thin skin between lifeless rock and lifeless, airless, freezing space. Then, on any timescale meaningful to "The Universe" individuals are tiny blips. Species are around longer, but not much. Vast majority of species are killed off by cosmic events regularly (we are on something like the 6th major extinction event, IIRC). All of humanities struggles with itself, with ignorance, all of it's artistic expression, have taken place within the context of that thin skin of warm fluid surrounding Earth over a shockingly short time frame. The best that can be said is that we now know the score: that "The Universe" is fascinating, vast, hostile, empty, silent, and our entire species is the rough equivalent not to a mewling baby, but rather to, perhaps, a single microscopic insect gestating in its egg, just beginning to break the shell and look around.
randcraw 11y ago
If there were a simple algorithm for intelligence, and assuming the presence of intelligence confers a competitive advantage, then why wouldn't natural selection have created many examples of it by now, after some billion years of multicellular life? Yet it hasn't. In fact intelligence manifests rarely in nature and only in long lived species with big complex brains which spend years educating their young to develop their cognitive skills. This implies that intelligence requires both a nontrivial nervous system and substantial nurturing thereof. Ergo, I conclude that intelligence itself is inherently complex and unlikely to arise simply or spontaneously.
- mpdehaan2 11y ago
  Every animal is intelligent. Using the google dictionary definition, "the ability to acquire and apply knowledge and skills." What they are are intelligent about varies, and the assumption that intelligence arises rarely is dangerous to ecological and environmental decision making. Rather, we don't know how to communicate with animals well and we don't think the same ways. http://www.nature.com/news/2010/100909/full/news.2010.458.html http://www.nature.com/news/2010/100909/full/news.2010.458.ht... http://scienceblogs.com/mixingmemory/2007/08/17/metatool-use-and-analogical-re/ http://scienceblogs.com/mixingmemory/2007/08/17/metatool-use... http://blogs.scientificamerican.com/running-ponies/catch-the-wave-decoding-the-prairie-doge28099s-contagious-jump-yips/ http://blogs.scientificamerican.com/running-ponies/catch-the... http://ngm.nationalgeographic.com/2015/05/dolphin-intelligence/foer-text http://ngm.nationalgeographic.com/2015/05/dolphin-intelligen...
- netheril96 11y ago
  > If there were a simple algorithm for intelligence, and assuming the presence of intelligence confers a competitive advantage, then why wouldn't natural selection have created many examples of it by now, after some billion years of multicellular life? A reasonable hypothesis would be that intelligence comes with high cost, such as high energy consumption, high heat production, longer period of infant stage, adaptation of body parts that will reduce their strength and interfere with their functionality, etc. All those negative effects more than offset the benefit of higher intelligence in the short term. Because natural selection does not plan but only pushes evolution toward local maximum points, most organism never evolves high intelligence.
esaym 11y ago
There really isn't an algorithm that can make algorithms, so no.
- gjm11 11y ago
  There are algorithms that make algorithms. (Even if we assume that the brain doesn't run on algorithms.) Genetic programming has made algorithms, for instance.
- joe_the_user 11y ago
  What do you mean? Generate random strings and feed them though a compiler. You have can create algorithms. That's not a very good algorithm but it shows algorithms that create algorithms exist. Now most modern AI algorithms, such as neural net, don't create algorithms (at least not directly), yes. But that might be indication that AI has further to go.
cafebeen 11y ago
A k-nearest neighbor classifier could be an example of a "simple" algorithm for intelligence, assuming of course, that you've collected a tremendous number of accurate (input, intelligent response) pairs!
garyrob 11y ago
For me, the question addressed by the article isn't even the interesting one. The author talks about "intelligence" as in how we process environmental information and perform reasoning tasks. From that perspective, it does seem obvious that some form of computer will one day be able to do it. I don't really care if the algorithms for doing so are simple or complicated. For me, the real question of interest is not about that kind of intelligence, but of "awareness". Computers may process environmental information and perform all sorts of logical operations based on it. But there is zero reason to think that any computer has any awareness whatsoever. It has no experience. There is something it is like to experience the color yellow, as opposed to merely classifying observed light as being of a certain wavelength; no computer has begun, in any way, to make that leap. It seems like a lot of people, including the author, don't even notice that this is a question. They seem to assume that computer-like information processing is all that happens in a human mind, and that awareness itself doesn't even exist as something to ponder. So they equate any doubt about whether a computer can do what we do as a form of vitalism. But with just a little bit of introspection, the fact that there is something it is like to experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength, or logically classifying it as having the attribute of "yellowness", is absolutely clear -- and absolutely fundamental to our existence as human beings (although many would assume that many other forms of life also have this basic ability to experience, if not the same ability to process information in computer-like ways). This is why, for me, the Turing Test is orthogonal to the question of whether a machine is actually conscious. I have no problem imagining that a software system, even on hardware that is basically the same as what we have now but bigger and faster, will one day be able to pass the Turing Test. But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does.
- Jabbles 11y ago
  Algorithms that can write heartfelt posts like yours will be developed one day. This is literally the point of Turing's thought experiment: if a machine gives you answers indistinguishable from a human, is it alive?
  - paulojreis 11y ago
jtth 11y ago
No.
jonmc12 11y ago
The author discusses concepts around intelligence without articulating what definition of intelligence he is adopting. When he discusses the innovation in understanding planetary motion, or chemical substances, it represents examples of where scientists could place observed data amongst a common frame of reference (ie, a meaningful definition to contextualizes observation leads the way to a theory which explains the data). A more meaningful direction for article, imo, would have been to adopt a concrete definition of intelligence, and then talk about how ferrets, chimps, babies and adults fall on this continuum. Perhaps, connectomics, molecular biology, psychology or neuro-anatomy help us approximate the 'atomic number', giving us the opportunity to speculate about the underlying equivalent of quantum mechanics. Like the article states, evidence is lacking, and we're somewhere between 1-100 nobel prizes away. But, the power of a concrete definition helps point to what we know we don't know, and discussion about whether the definition of intelligence itself needs to be re-framed to answer a question like this.
kevinalexbrown 11y ago
I enjoy the planetary motion analogy immensely. However, a curious complexity is evident in biological systems - it's physics imbued with 'meaning'. A cell that behaves one way instead of another will die. This is true from e. coli up to skin cells. I suspect this adds a lot of complexity to the ultimate 'algorithm' of intelligence. In fact, it's probably prudent to stipulate whether the goal is to specify the developmental algorithm that gives rise to intelligence, or intelligence itself. The generating process might be a lot more simple than the finished product, in the same way that the output of a pseudorandom number generator is very complex in terms of entropy (but not kolmogorov complexity, I guess), but the generating algorithm is very simple. Or the Rado graph, which is sorta maximally complex in the sense that it contains all finite and countably infinite graphs as induced subgraphs, yet it has a simple generating scheme. As a last example, consider No Man's Sky - relatively simple algorithm compared to the astonishing complexity of the worlds. I do believe there is a relatively simple high-level description of neural development. But it's curious how it's relatively robust to genetic manipulations. I remember early in graduate school listening to a lecture about a particular mouse model for autism, caused by just one gene. The lecturer excitedly told us that it had a very high rate of behavioral manipulation - 75% or so. Not coming from a mouse-model background I was astonished that it was 'only' 75%. What happens in the other 25% - the gene doesn't just magically reappear, there are other compensatory mechanisms. What drives those? I suspect that's the more abstract level of description that a simple algorithm might describe. https://en.wikipedia.org/wiki/Rado_graph https://en.wikipedia.org/wiki/Rado_graph
- deleted 11y ago
  [deleted]
compbio 11y ago
Since I did not see it in the article or in the comments here: Intelligence is compression / dimensionality reduction. Finding the essential parts of a problem/object/concept and creating a code book for it. The better you can compress things, the better you understand them. If you know that the left wing of a plane is the same as the right wing of a plane, but rotated/flipped, then you would not need to store (redundant) information about both wings. The upper bound to an intelligent action is the Kolmogorov complexity (upper bound to compression) of the problem it aims to solve. http://www.hutter1.net/ai/pfastprg.htm http://www.hutter1.net/ai/pfastprg.htm (The Fastest and Shortest Algorithm for All Well-Defined Problems)
umutisik 11y ago
Another imperfect analogy: say someone (maybe from an earlier time) was looking to reverse engineer a modern CPU but could not see what was going on inside it very well. Surely the whole thing would be very confusing. The CPU has many many clever designs based on the needs of people who buy it and use it, but the whole thing is based on how you can make AND/NOT using transistors. Maybe if the person was to discover this, then they could start getting past the complexity of reconstructing some of the mechanisms of the CPU; and they could start making their own CPU for their own needs.
- seiji 11y ago
  Take it one step further: say person from the past sees OS X running on a computer. They take the CPU and try to reverse-engineer OS X just from the CPU. Good luck.
api 11y ago
I think a strong argument for 'no' comes from the No Free Lunch (NFL) theorems: https://en.wikipedia.org/wiki/No_free_lunch_theorem https://en.wikipedia.org/wiki/No_free_lunch_theorem The NFL theorems have been misused by advocates of 'intelligent design' -- they do not factor against evolution since evolution doesn't demand a global maximum, only a local one. (There are other reasons but this is the most fundamental.) But the theorems themselves are incredibly interesting. In short they show that averaged over the space of all possible search landscapes, no search algorithm performs better than any other. Obviously the universe does not provide "all possible search landscapes," since the universe has structure. Some search landscape structures and meta-structures are pathological and rare in nature. But nature does provide a huge diversity of them. The NFL theorem is to me powerful evidence that something as general as human or even animal intelligence must be a superposition of multiple "algorithms" rather than just one. It also, I think, explains why all single-algorithm or single-approach methods of AI end up being domain specific. Obviously they'll be domain specific -- they only work against search landscapes with a certain structure! The NFL theorems are a big reason I am a short term AI skeptic. I don't think human-level or beyond AI is impossible, but I think we are quite far from realizing it. I'll become more optimistic when AI researchers start studying biology more closely and deeply, since biological systems are the only existing examples of truly intelligent systems.
- rsaarelm 11y ago
  Solomonoff induction seems to be an useful innate bias for the current approaches at general intelligence. It's basically a formalization of Occam's razor that states that instead of being completely random, the surrounding environment is mostly lawful and simple, and therefore most of the processes in it can be described by algorithms with short rather than long source code. Hutter has written a bit about NFL: http://arxiv.org/pdf/1111.3846.pdf http://arxiv.org/pdf/1111.3846.pdf http://arxiv.org/pdf/1105.5721.pdf http://arxiv.org/pdf/1105.5721.pdf
  - api 11y ago
    Thanks, those look interesting. Will give them a read when I have time. My naive reaction though is: a lawful universe does not necessarily imply well-behaved fitness landscapes. Many lawful and even very simple processes give rise to chaotic and complex results.
jokoon 11y ago
I believe there is research to be done in hardware that resembles neuron networks. Im sure one could build a dedicated chip with rewirable programmable gates. I still wonder if gpus are really efficient at simulating NNs. It might get very expensive, but like he said, im optimist that replicating nn in hardware could be the way to go. i think its a problem where classic turing machines are too different from networks of neurons.
astazangasta 11y ago
I have a question: why do we even want a general artificial intelligence? Isn't human thought good enough? I can only think of one reason to make such a thing: to enslave it and use it to control other humans.
gwern 11y ago
Jacob Cannell has a recent blog post that goes into more depth about the neurobiological aspects of the master-trick argument: http://lesswrong.com/lw/md2/the_brain_as_a_universal_learning_machine/ http://lesswrong.com/lw/md2/the_brain_as_a_universal_learnin... (also good is http://lesswrong.com/lw/meo/analogical_reasoning_and_creativity/ http://lesswrong.com/lw/meo/analogical_reasoning_and_creativ... )
escherplex 11y ago
First define 'intelligence'. OED: 1) faculty of understanding, 2) quickness, 5) knowledge; exchange of knowledge. To abstract from Chalmers: cognitive software supervening on a conformal neural hardware substrate. OK but in everyday empirical-focused psychology, how are subjects tested for 'intelligence'? My brother and his colleagues draw from Gardner's eight 'abilities': rhythmic (musical), visual (spatial), linguistic, logical (mathematical), bodily (kinesthetic), interpersonal, intrapersonal, and naturalistic. IE, there are subjective elements in evaluating 'intelligence' plus 'fuzzy logic' only supplies a useful fiction for quantifying the subjective. Now this article appears preoccupied with mapping-out neural hardware specs in its emphasis on processing capacity, memory, clock speed and whatnot. Would a savant mentally capable of calculating pi to a zillion places but incapable of communicating results be judged 'intelligent'? MRI maps of mental processing only would suggest such. Would a quantum MB capable of running MATLAB at warp speed but unable to render output in an ascertainable form be judged well-scripted? Not! Even a perusal of the current WAIS-R manual (no particulars since I'm not predisposed to be shot) exhibits classes of both objective testing and tasks which would be categorized also as measures of interpersonal engagement (Who was famous for ...). Would the Ava character in the movie 'Ex Machina' be judged consciously 'intelligent'? Consensus probably would be 'yes' since both her subjective and objective 3D GUI seemed correct by human standards towards the end. [wonder if anyone who saw the movie concluded that the affective agent of Ava’s cognitive development harkened back to John Locke's premise that percepts functioned as something analogous to self-extracting .zip files which imposed real cognitive patterns on a subject's 'tabula rosa', not requiring any native cognitive modeling]. But would Ava really 'understand' anything or just be a 'Searle's Chinese box'?
- escherplex 11y ago
  strong-AI crowd eh?
RangerScience 11y ago
Arguably, yes: http://phys.org/news/2013-04-emergence-complex-behaviors-causal-entropic.html http://phys.org/news/2013-04-emergence-complex-behaviors-cau... Basically, if you act to maximize the adjacency of possible futures, you act intelligently, and an agent that does this can dynamically figure out the inverted pendulum, tool use, cooperation, and cooperative tool use. (By "dynamically" I mean "without prior instruction".) However, in order to use this process, the agent must also have a way to predict the future states of a system. When you stop and think about what THAT part entails, shit starts to get even more interesting. PS - It gets even more interesting when you pick up on how this mimics entropy, and then this: https://www.quantamagazine.org/20140122-a-new-physics-theory-of-life/ https://www.quantamagazine.org/20140122-a-new-physics-theory...

129 points by rcoppolo 11y ago

tl;dr - "We don't know[but please read my blog]."

>So 125 million base pairs is equivalent to 250 million bits of information. That's the genetic difference between humans and chimps! I don't mean to nitpick, but epigenetics is a thing. Also... how up-to-date on the research literature is this essay supposed to be?

TheOtherHobbes 11y ago
The information isn't just in the base pairs. It's in the entire evolved ecology (with optional industrial add-on pack) that makes the base pairs do something useful. How much of that could you take away and still get a human brain?
- e_modad 11y ago
  Craig Venter says here[1] that Franz Och (of Google Translate fame) has done some analysis for his new company Human Longevity Inc that shows that about 50% of the genome is responsible for the brain. So the number is quite big and much larger than I would have thought. Excited to see more results from HLI. Comment at 13:12 [1] https://youtu.be/D3nIIKwwiLc?t=11m43s https://youtu.be/D3nIIKwwiLc?t=11m43s
  - bohm 11y ago
    Which is interesting considering we share about 85% of our genome with Zebra Fish.
    - e_modad 11y ago
      Source? That sounds a little high.
      - bohm 11y ago
        From memory, but maybe a slight overstatement: http://www.sci-news.com/genetics/article01036.html http://www.sci-news.com/genetics/article01036.html 70 per cent of protein-coding human genes are related to genes found in the zebrafish (Danio rerio), and 84 per cent of genes known to be associated with human disease have a zebrafish counterpart.

cousin_it 11y ago

Given unlimited computing power, AIXItl [1] is a pretty simple algorithm that can provably solve all problems, in some sense, at least as well as any other algorithm. The idea is to simply dovetail over all possible algorithms and select the ones that fit observations best. That includes humans, if you believe as I do that humans are computable. With limited computing power, it's likely that the best algorithms for many different problems (possibly including "intelligence" however you define it) won't be simple, just like the best known algorithms for integer multiplication [2] aren't simple. In particular, AIXI variants will be hard to approximate, precisely because they dovetail over all possible algorithms. That said, it's very likely that the best algorithms for intelligence will be simpler and faster than humans, because humans are the stupidest possible creatures that can build a civilization (otherwise it would've happened earlier in our evolution). [1] http://www.hutter1.net/ai/paixi.htm http://www.hutter1.net/ai/paixi.htm [2] https://en.wikipedia.org/wiki/F%C3%BCrer%27s_algorithm https://en.wikipedia.org/wiki/F%C3%BCrer%27s_algorithm

eli_gottlieb 11y ago
>Given unlimited computing power, AIXItl [1] is a pretty simple algorithm that can provably solve all problems at least as well, in some sense, as any other algorithm. Technically, AIXI_{tl} solves all problems at least as well as any other algorithm in a given order of asymptotic time and length, those being the t and l. The problem is that it has a "trivial additive constant" which will make it take longer than the lifetimes of many stars to achieve its much-vaunted "optimal" asymptotic performance. In short, AIXI is the most utterly brute-force kind of "intelligence" you could possibly build, whose notion of "scaled down" bounded-rational inference is still astronomically intractable. It basically just amounts to throwing computing power at the problem, more computing power than anyone actually has. Mind, Schmidhuber and Hutter still get massive props for managing to cut out the philoso-wank that usually accompanies discussions of cognition and instead saying, "Let's just pose a very general inference problem and write an algorithm that solves it."
- cousin_it 11y ago
  Yeah, that's why I said "given unlimited computing power". But theoretical work sometimes has a way of becoming practical and scary. Check out the work of Joel Veness on Monte Carlo AIXI, which learned to play Pac-Man on a single desktop computer, presumably faster than "the lifetimes of many stars".

meeper16 11y ago

It has to do with pattern matching. In particular, comparing vectors with one another for similarity or dissimilarity.

knodi123 11y ago
define "vector" in this context

kazinator 11y ago

Yes, there is a simple algorithm for intelligence. It's just a few lines of code. However, it requires a terabyte-sized table of data which has yet to be filled in.

knodi123 11y ago
there's a simple algorithm for running the latest version of windows. It's only a few lines of code, and a hundred gigs of compressed C++.

bohm 11y ago

This ignores that genetic code is meaningless without the translation machinery (ribosomes and the entire physical world of physics and chemistry). Genetic code without the translation machinery and the physical world it is translated into is meaningless. A multi-cellular embodied intelligence can acquire information about the patterns of the physical world by growing in it - the code is compact because much of the necessary information can be acquired during embodied growth. A disembodied intelligence such as AIs can't rely on external storage and acquisition of information during a cellular growth stage, and thus must likely be considerably more complex from the get-go.

rndn 11y ago
It also ignores the possibilities (1), that cells and tissues are extremely complex (and thus require a lot of genetic code), while the overall algorithm the cells and tissues perform might still be very simple, and conversely (2), that chimpanzees might be already extremely intelligent and the difference is rather a matter of embodiment (in particular the ability to produce language), and perhaps minor tweaks to neurons, to the cortex and/or other brain regions.
- bohm 11y ago
  According to the Santiago Theory of Cognition "Living systems are cognitive systems, and living as a process is a process of cognition. This statement is valid for all organisms, with or without a nervous system." https://en.wikipedia.org/wiki/Santiago_Theory_of_Cognition https://en.wikipedia.org/wiki/Santiago_Theory_of_Cognition If this were a correct interpretation, even Bacteria could be seen as already incredibly intelligent. Going Zero to One (dead matter -> prokaryotes) is a significantly bigger step than going One to Five (prokaryotes -> humans).
  - Retric 11y ago
    Careful, modern prokaryotes also have billions of years of evolution behind them. The first organism where likely inefficient and would not last in our current hyper competitive ecosystems. Arguably, the first organism where really more just free floating proteins without even a cell wall.

ThomPete 11y ago

Wouldn't there have to be? Given that we evolved from much simpler organisms.

UhUhUhUh 11y ago

I hold the same belief. I also think that we don't pay enough attention to the form of processes whereas nature seems to place lots of emphasis on it. In fact, the more it goes, the more a form factor appears to be somehow determinant. Molecular biology with enzymes, genetics with histones, neurobiology with firing patterns, etc. We can see that now because we have access to finer grained observation. Form could be a higher-order type of content and could also be approximated with an algorithm. This all feeds in a previous discussion about intuition. Top-down or bottom-up?

300bps 11y ago

In the 16th century only a foolish optimist could have imagined that all these objects' motions could be explained by a simple set of principles. But in the 17th century Newton formulated his theory of universal gravitation, which not only explained all these motions, but also explained terrestrial phenomena such as the tides and the behaviour of Earth-bound projecticles. It turns out that Newton's ideas on gravity are a mathematical approximation that at least ignored relativistic effects. See: http://physics.stackexchange.com/questions/52165/newtonian-gravity-vs-general-relativity-exactly-how-wrong-is-newton http://physics.stackexchange.com/questions/52165/newtonian-g... Newton's gravity was a theory about a single fundamental force - and it turned out to be not 100% accurate. Rather than be an example of how intelligence might be similarly possible to explain with a simple theory, I think it offers a perfect counter-example of why it almost certainly cannot.

habitue 11y ago
Comets, and the planets and the tides, and apples falling from trees were found to be caused by the same underlying phenomena, despite seeming very different. Newton's insight that they were the same was not overturned by relativity, so the point stands quite well.
- 300bps 11y ago
  Newton's insight that they were the same was not overturned by relativity I almost feel like you're replying to someone else because I never said what you are contradicting. I merely said that Newton's gravity is a simplification that doesn't take all variables into account. So to point to Newton's gravity as an example of, "Sometimes simple things can be right!" actually makes it more of a counter-example.
  - habitue 11y ago
    I was pointing out that the complex explanation is that comets have their own driving force, and tides have a separate driving force, and planets have their own reason for moving as they do, etc. Compared to that kind of hodge-podge, relativity is still a really simple theory. While relativity increased the complexity of the explanation of gravity, it didn't bring it anywhere near the level of complexity that might have been assumed before Newton. The author's argument is that our intelligence is likely not solely possible by the interaction of a large number of separate principles, but rather by some simple principle that manifests itself in different ways. I think if we find some simple principle that explains intelligence, and then later have to make a correction to the math of that principle, the resulting explanation will still be simpler than the idea that intelligence is made up of many different principles all of which have to be present.

emergentcypher 11y ago

Something something law of headlines something answer is "no".

thomasfoster96 11y ago

I'd think a 'simple' algorithm for intelligence would be even more distant than artificial general intelligence itself.

knodi123 11y ago
Simplicity is always only a little ways off. If I sat you next to god and told you to pair program and write an OS equivalent to Windows XP, with you driving the keyboard- you'd be sitting there for quite a while. But if I told you guys to pair program an AGI, you could probably finish in a day or so. A simple algorithm is a matter of finding just one or two basic discoveries. It could happen tomorrow! It probably won't, but that's the nature of discovery.
- thomasfoster96 11y ago
  Well, supposing I'm not programming with god, rather I'm with a bunch of programmer friends, I still doubt the first artificial general intelligence (assuming we happen to be the people who make it) will be a beautifully simple piece of software. It'll probably be horribly hacked together and hard to maintain, and only in time will a simpler way to do things emerge. For example, I'd reckon we'll find that a lot of things that today only make sense to solve with a neural net are actually perfectly representable using a simple(ish) algorithm, but it's unlikely we'll find that out until after we create an artificial general intelligence and know more about the problem of intelligence. In other words, we'll solve the problem, and then realise how much easier it could have been.
  - knodi123 11y ago
    https://xkcd.com/224/ https://xkcd.com/224/

paulojreis 11y ago

> "I believe it's not in serious doubt that an intelligent computer is possible" - Yes, of course it is possible, if we change the meaning of "intelligent" to fit what a computer does. :) Generally, I think the problem here - as in many other situations - starts with the lack of rigor that computer science has when "naming" fields or, most exactly, appropriating concepts from other fields. * Computer Vision: "vision" has a defined meaning, even "organic-related". Signal processing and a bunch of algorithms aren't exactly what vision is and what we define as "vision"; * Machine learning: does the computer really learns, as understood by the common definition of "learning"? * Context awareness: if we consider common definitions, neither a few sensor readings are "context", nor acting according to said readings is "awareness". This might sound like nitpicking, but I truly think it's a real problem. Talking about vision, learning or intelligence put's the stake at what people or the general scientific community commonly perceive as those concepts. What's most frustrating to me is the contrast. Computer science research is thorough; why are we not thorough on naming our fields of study? Why do we appropriate concepts by "analogy", knowingly that analogies aren't truly exact?

habitue 11y ago
> Yes, of course it is possible, if we change the meaning of "intelligent" to fit what a computer does. :) I don't think that's what he's doing. Otherwise he could just define intelligence to mean what computers do now. Clearly he has at least roughly the same idea of intelligence you or I do. > "I believe it's not in serious doubt that an intelligent computer is possible" Do you have an objection to this specific statement when taken to mean general human-like intelligence being exhibited by computers? I have not heard anyone defend the opposite viewpoint often, and I'm interested if there are any compelling arguments why it's not possible.
- turkeysandwich 11y ago
  Any algorithms we use to simulate intelligence with machines are excessively wasteful. There's not enough energy to simulate human intelligence at the same speed that humans think, without just essentially producing biological computers. And that's more of bio-engineering than anything relating to computing.

mathgenius 11y ago

> I believe it's not in serious doubt that an intelligent computer is possible - although it may be extremely complicated, and perhaps far beyond current technology - and current naysayers will one day seem much like the vitalists. Physicists (and this guy is one of them) certainly do have a robust set of taboos against considering consciousness as a quantum phenomena (whatever that might mean.) It's unfortunate because there is a huge resurgence going on in the physics of quantum many-body entanglement, quantum computers, foundations of quantum, etc. Alot of this meandering was just sidelined in the 1940s when world affairs intervened. However, in light of these new discoveries it's time to revisit the old vitalist arguments. Imho.

TTPrograms 11y ago
If consciousness is a quantum phenomenon it has coherence scales orders of magnitude beyond what we can achieve in labs at ~1 Kelvin. That is, it's pretty unlikely that quantum effects are having macro scale effects on the brain.
- mathgenius 11y ago
  Yes, this is certainly the prevailing opinion among practicing physicists. I would like to point out though, that this is a comparison between a few years of tinkering in the lab with billions of years of evolution.
  - TTPrograms 11y ago
    That's a good point. We should probably not trust this "lab tinkering" because it's so new - nature already figured everything out already. I'll go back to hunting and gathering now...
habitue 11y ago
Let's say, for the sake of argument, that intelligence requires quantum effects to work. I don't see how going back to vitalist arguments will aid us in any way. Researching the phenomena, following evidence, making inferences from that evidence... that's how progress on that subject would be made. Not by going back to arguments made by people with less information than we have now. Specifically, if we follow the evidence, and find that somehow, beyond all odds, the vitalists were correct (once you add "quantumness!" to the mix), then it's clearly only by lucky guess they were right. It would be something we could look back at and say "wow, that's an interesting coincidence" rather than "wow, those vitalists had it right all along, we should have paid closer attention".

amelius 11y ago

There probably is a very simple algorithm for intelligence. It is just way to expensive to run it. (The reason is very simple, namely that the rules for the universe are probably very simple.)

norea-armozel 11y ago

Just from skimming the article it seems to me the problem with AI and probably just the whole matter of what people define as intelligence is that it assumes that the universe itself is not intelligent. Maybe the universe itself is intelligent but that consciousness is some other phenomena that gives intelligence an individual characteristic? It's just a random thought, I don't really know which way to go with it.

davesque 11y ago
I can relate. While I consider myself an agnostic with a slight atheist streak, I still like to muse on these sorts of thoughts occasionally. The universe does seem somehow innately intelligent and alive. Everything is in motion. Everything appears to have some kind of structure. Gotta be careful not to fall back on the "G" word though (not that you did) :).
- norea-armozel 11y ago
  Personally, I think there might be a Creator, but that doesn't seem to be relevant as to the problem of consciousness. To me, when I see that physical processes act in a manner that is complex then it indicates that what we're trying to call intelligence is really just form of stateful structure. And the rest of intelligence is something else entirely different (and probably unrelated).
- javajosh 11y ago
  "The Universe" is huge and empty and completely hostile to life. Most matter is in stars which are not empty and completely hostile to life. (And which are huge compared to us, but which are like specks of dust compared to "The Universe"). Our planet is a speck of dust rotating at a fortunate distance around another speck of (shining) dust. By mass, the majority of the planet is useless to life. Everything alive on Earth exists in a very thin skin between lifeless rock and lifeless, airless, freezing space. Then, on any timescale meaningful to "The Universe" individuals are tiny blips. Species are around longer, but not much. Vast majority of species are killed off by cosmic events regularly (we are on something like the 6th major extinction event, IIRC). All of humanities struggles with itself, with ignorance, all of it's artistic expression, have taken place within the context of that thin skin of warm fluid surrounding Earth over a shockingly short time frame. The best that can be said is that we now know the score: that "The Universe" is fascinating, vast, hostile, empty, silent, and our entire species is the rough equivalent not to a mewling baby, but rather to, perhaps, a single microscopic insect gestating in its egg, just beginning to break the shell and look around.

randcraw 11y ago

If there were a simple algorithm for intelligence, and assuming the presence of intelligence confers a competitive advantage, then why wouldn't natural selection have created many examples of it by now, after some billion years of multicellular life? Yet it hasn't. In fact intelligence manifests rarely in nature and only in long lived species with big complex brains which spend years educating their young to develop their cognitive skills. This implies that intelligence requires both a nontrivial nervous system and substantial nurturing thereof. Ergo, I conclude that intelligence itself is inherently complex and unlikely to arise simply or spontaneously.

mpdehaan2 11y ago
Every animal is intelligent. Using the google dictionary definition, "the ability to acquire and apply knowledge and skills." What they are are intelligent about varies, and the assumption that intelligence arises rarely is dangerous to ecological and environmental decision making. Rather, we don't know how to communicate with animals well and we don't think the same ways. http://www.nature.com/news/2010/100909/full/news.2010.458.html http://www.nature.com/news/2010/100909/full/news.2010.458.ht... http://scienceblogs.com/mixingmemory/2007/08/17/metatool-use-and-analogical-re/ http://scienceblogs.com/mixingmemory/2007/08/17/metatool-use... http://blogs.scientificamerican.com/running-ponies/catch-the-wave-decoding-the-prairie-doge28099s-contagious-jump-yips/ http://blogs.scientificamerican.com/running-ponies/catch-the... http://ngm.nationalgeographic.com/2015/05/dolphin-intelligence/foer-text http://ngm.nationalgeographic.com/2015/05/dolphin-intelligen...
netheril96 11y ago
> If there were a simple algorithm for intelligence, and assuming the presence of intelligence confers a competitive advantage, then why wouldn't natural selection have created many examples of it by now, after some billion years of multicellular life? A reasonable hypothesis would be that intelligence comes with high cost, such as high energy consumption, high heat production, longer period of infant stage, adaptation of body parts that will reduce their strength and interfere with their functionality, etc. All those negative effects more than offset the benefit of higher intelligence in the short term. Because natural selection does not plan but only pushes evolution toward local maximum points, most organism never evolves high intelligence.

esaym 11y ago

There really isn't an algorithm that can make algorithms, so no.

gjm11 11y ago
There are algorithms that make algorithms. (Even if we assume that the brain doesn't run on algorithms.) Genetic programming has made algorithms, for instance.
joe_the_user 11y ago
What do you mean? Generate random strings and feed them though a compiler. You have can create algorithms. That's not a very good algorithm but it shows algorithms that create algorithms exist. Now most modern AI algorithms, such as neural net, don't create algorithms (at least not directly), yes. But that might be indication that AI has further to go.

cafebeen 11y ago

A k-nearest neighbor classifier could be an example of a "simple" algorithm for intelligence, assuming of course, that you've collected a tremendous number of accurate (input, intelligent response) pairs!

garyrob 11y ago

For me, the question addressed by the article isn't even the interesting one. The author talks about "intelligence" as in how we process environmental information and perform reasoning tasks. From that perspective, it does seem obvious that some form of computer will one day be able to do it. I don't really care if the algorithms for doing so are simple or complicated. For me, the real question of interest is not about that kind of intelligence, but of "awareness". Computers may process environmental information and perform all sorts of logical operations based on it. But there is zero reason to think that any computer has any awareness whatsoever. It has no experience. There is something it is like to experience the color yellow, as opposed to merely classifying observed light as being of a certain wavelength; no computer has begun, in any way, to make that leap. It seems like a lot of people, including the author, don't even notice that this is a question. They seem to assume that computer-like information processing is all that happens in a human mind, and that awareness itself doesn't even exist as something to ponder. So they equate any doubt about whether a computer can do what we do as a form of vitalism. But with just a little bit of introspection, the fact that there is something it is like to experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength, or logically classifying it as having the attribute of "yellowness", is absolutely clear -- and absolutely fundamental to our existence as human beings (although many would assume that many other forms of life also have this basic ability to experience, if not the same ability to process information in computer-like ways). This is why, for me, the Turing Test is orthogonal to the question of whether a machine is actually conscious. I have no problem imagining that a software system, even on hardware that is basically the same as what we have now but bigger and faster, will one day be able to pass the Turing Test. But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does.

Jabbles 11y ago
Algorithms that can write heartfelt posts like yours will be developed one day. This is literally the point of Turing's thought experiment: if a machine gives you answers indistinguishable from a human, is it alive?
- paulojreis 11y ago

jtth 11y ago

No.

jonmc12 11y ago

The author discusses concepts around intelligence without articulating what definition of intelligence he is adopting. When he discusses the innovation in understanding planetary motion, or chemical substances, it represents examples of where scientists could place observed data amongst a common frame of reference (ie, a meaningful definition to contextualizes observation leads the way to a theory which explains the data). A more meaningful direction for article, imo, would have been to adopt a concrete definition of intelligence, and then talk about how ferrets, chimps, babies and adults fall on this continuum. Perhaps, connectomics, molecular biology, psychology or neuro-anatomy help us approximate the 'atomic number', giving us the opportunity to speculate about the underlying equivalent of quantum mechanics. Like the article states, evidence is lacking, and we're somewhere between 1-100 nobel prizes away. But, the power of a concrete definition helps point to what we know we don't know, and discussion about whether the definition of intelligence itself needs to be re-framed to answer a question like this.

kevinalexbrown 11y ago

I enjoy the planetary motion analogy immensely. However, a curious complexity is evident in biological systems - it's physics imbued with 'meaning'. A cell that behaves one way instead of another will die. This is true from e. coli up to skin cells. I suspect this adds a lot of complexity to the ultimate 'algorithm' of intelligence. In fact, it's probably prudent to stipulate whether the goal is to specify the developmental algorithm that gives rise to intelligence, or intelligence itself. The generating process might be a lot more simple than the finished product, in the same way that the output of a pseudorandom number generator is very complex in terms of entropy (but not kolmogorov complexity, I guess), but the generating algorithm is very simple. Or the Rado graph, which is sorta maximally complex in the sense that it contains all finite and countably infinite graphs as induced subgraphs, yet it has a simple generating scheme. As a last example, consider No Man's Sky - relatively simple algorithm compared to the astonishing complexity of the worlds. I do believe there is a relatively simple high-level description of neural development. But it's curious how it's relatively robust to genetic manipulations. I remember early in graduate school listening to a lecture about a particular mouse model for autism, caused by just one gene. The lecturer excitedly told us that it had a very high rate of behavioral manipulation - 75% or so. Not coming from a mouse-model background I was astonished that it was 'only' 75%. What happens in the other 25% - the gene doesn't just magically reappear, there are other compensatory mechanisms. What drives those? I suspect that's the more abstract level of description that a simple algorithm might describe. https://en.wikipedia.org/wiki/Rado_graph https://en.wikipedia.org/wiki/Rado_graph

deleted 11y ago
[deleted]

compbio 11y ago

Since I did not see it in the article or in the comments here: Intelligence is compression / dimensionality reduction. Finding the essential parts of a problem/object/concept and creating a code book for it. The better you can compress things, the better you understand them. If you know that the left wing of a plane is the same as the right wing of a plane, but rotated/flipped, then you would not need to store (redundant) information about both wings. The upper bound to an intelligent action is the Kolmogorov complexity (upper bound to compression) of the problem it aims to solve. http://www.hutter1.net/ai/pfastprg.htm http://www.hutter1.net/ai/pfastprg.htm (The Fastest and Shortest Algorithm for All Well-Defined Problems)

umutisik 11y ago

Another imperfect analogy: say someone (maybe from an earlier time) was looking to reverse engineer a modern CPU but could not see what was going on inside it very well. Surely the whole thing would be very confusing. The CPU has many many clever designs based on the needs of people who buy it and use it, but the whole thing is based on how you can make AND/NOT using transistors. Maybe if the person was to discover this, then they could start getting past the complexity of reconstructing some of the mechanisms of the CPU; and they could start making their own CPU for their own needs.

seiji 11y ago
Take it one step further: say person from the past sees OS X running on a computer. They take the CPU and try to reverse-engineer OS X just from the CPU. Good luck.

api 11y ago

I think a strong argument for 'no' comes from the No Free Lunch (NFL) theorems: https://en.wikipedia.org/wiki/No_free_lunch_theorem https://en.wikipedia.org/wiki/No_free_lunch_theorem The NFL theorems have been misused by advocates of 'intelligent design' -- they do not factor against evolution since evolution doesn't demand a global maximum, only a local one. (There are other reasons but this is the most fundamental.) But the theorems themselves are incredibly interesting. In short they show that averaged over the space of all possible search landscapes, no search algorithm performs better than any other. Obviously the universe does not provide "all possible search landscapes," since the universe has structure. Some search landscape structures and meta-structures are pathological and rare in nature. But nature does provide a huge diversity of them. The NFL theorem is to me powerful evidence that something as general as human or even animal intelligence must be a superposition of multiple "algorithms" rather than just one. It also, I think, explains why all single-algorithm or single-approach methods of AI end up being domain specific. Obviously they'll be domain specific -- they only work against search landscapes with a certain structure! The NFL theorems are a big reason I am a short term AI skeptic. I don't think human-level or beyond AI is impossible, but I think we are quite far from realizing it. I'll become more optimistic when AI researchers start studying biology more closely and deeply, since biological systems are the only existing examples of truly intelligent systems.

rsaarelm 11y ago
Solomonoff induction seems to be an useful innate bias for the current approaches at general intelligence. It's basically a formalization of Occam's razor that states that instead of being completely random, the surrounding environment is mostly lawful and simple, and therefore most of the processes in it can be described by algorithms with short rather than long source code. Hutter has written a bit about NFL: http://arxiv.org/pdf/1111.3846.pdf http://arxiv.org/pdf/1111.3846.pdf http://arxiv.org/pdf/1105.5721.pdf http://arxiv.org/pdf/1105.5721.pdf
- api 11y ago
  Thanks, those look interesting. Will give them a read when I have time. My naive reaction though is: a lawful universe does not necessarily imply well-behaved fitness landscapes. Many lawful and even very simple processes give rise to chaotic and complex results.

jokoon 11y ago

I believe there is research to be done in hardware that resembles neuron networks. Im sure one could build a dedicated chip with rewirable programmable gates. I still wonder if gpus are really efficient at simulating NNs. It might get very expensive, but like he said, im optimist that replicating nn in hardware could be the way to go. i think its a problem where classic turing machines are too different from networks of neurons.

astazangasta 11y ago

I have a question: why do we even want a general artificial intelligence? Isn't human thought good enough? I can only think of one reason to make such a thing: to enslave it and use it to control other humans.

gwern 11y ago

Jacob Cannell has a recent blog post that goes into more depth about the neurobiological aspects of the master-trick argument: http://lesswrong.com/lw/md2/the_brain_as_a_universal_learning_machine/ http://lesswrong.com/lw/md2/the_brain_as_a_universal_learnin... (also good is http://lesswrong.com/lw/meo/analogical_reasoning_and_creativity/ http://lesswrong.com/lw/meo/analogical_reasoning_and_creativ... )

escherplex 11y ago

First define 'intelligence'. OED: 1) faculty of understanding, 2) quickness, 5) knowledge; exchange of knowledge. To abstract from Chalmers: cognitive software supervening on a conformal neural hardware substrate. OK but in everyday empirical-focused psychology, how are subjects tested for 'intelligence'? My brother and his colleagues draw from Gardner's eight 'abilities': rhythmic (musical), visual (spatial), linguistic, logical (mathematical), bodily (kinesthetic), interpersonal, intrapersonal, and naturalistic. IE, there are subjective elements in evaluating 'intelligence' plus 'fuzzy logic' only supplies a useful fiction for quantifying the subjective. Now this article appears preoccupied with mapping-out neural hardware specs in its emphasis on processing capacity, memory, clock speed and whatnot. Would a savant mentally capable of calculating pi to a zillion places but incapable of communicating results be judged 'intelligent'? MRI maps of mental processing only would suggest such. Would a quantum MB capable of running MATLAB at warp speed but unable to render output in an ascertainable form be judged well-scripted? Not! Even a perusal of the current WAIS-R manual (no particulars since I'm not predisposed to be shot) exhibits classes of both objective testing and tasks which would be categorized also as measures of interpersonal engagement (Who was famous for ...). Would the Ava character in the movie 'Ex Machina' be judged consciously 'intelligent'? Consensus probably would be 'yes' since both her subjective and objective 3D GUI seemed correct by human standards towards the end. [wonder if anyone who saw the movie concluded that the affective agent of Ava’s cognitive development harkened back to John Locke's premise that percepts functioned as something analogous to self-extracting .zip files which imposed real cognitive patterns on a subject's 'tabula rosa', not requiring any native cognitive modeling]. But would Ava really 'understand' anything or just be a 'Searle's Chinese box'?

escherplex 11y ago
strong-AI crowd eh?

RangerScience 11y ago

Arguably, yes: http://phys.org/news/2013-04-emergence-complex-behaviors-causal-entropic.html http://phys.org/news/2013-04-emergence-complex-behaviors-cau... Basically, if you act to maximize the adjacency of possible futures, you act intelligently, and an agent that does this can dynamically figure out the inverted pendulum, tool use, cooperation, and cooperative tool use. (By "dynamically" I mean "without prior instruction".) However, in order to use this process, the agent must also have a way to predict the future states of a system. When you stop and think about what THAT part entails, shit starts to get even more interesting. PS - It gets even more interesting when you pick up on how this mimics entropy, and then this: https://www.quantamagazine.org/20140122-a-new-physics-theory-of-life/ https://www.quantamagazine.org/20140122-a-new-physics-theory...