5 ms·

> If you go out and actually study the only known systems which learn effectively (ie., animals), one does not find the need for universal approximator theorems

by shawnz 5y ago

> If you go out and actually study the only known systems which learn effectively (ie., animals), one does not find the need for universal approximator theorems in explaining their capacities. What's the explanation for their capacities, then?

mjburgess 5y ago
direct, theory-laden, causal contact and pro-active engagement with their environments... eg., taking the stick out of the water. "data" (even from an animal's pov) is basically, by nature, ambiguous. Measurement is an event in the world (eg., light hitting the eye) which isn't somehow unambiguously informative. To over come this problem, basically, animals move stuff. The adaptable motor cortex of the most intelligent animals, therefore, isnt something to be tacked-on to "intelligence", it's the precondition for it. Glibly: tools before thoughts. Reason needs content to operate on, the content of our thoughts is built via our (sensory-)motor systems. The idea that we need ever more theoretically powerful models of reasoning here is a misdirection -- it misses that the heart of everything we know arrives via some form of repeated experiment. Every more automated "pure reasoning" either via statistics or symbolically, is always just a means of juicing the data we provide the system. Useful as a technology, but not as a genuine learning system. It will never have the means of resolving the many ambiguities within data itself. In the case, for example, of NLP -- the structure of 1 trillion documents will never enable a machine to answer the question "what do you like about what i'm wearing?" -- because (1) the machine isnt here with me; (2) i am asking for its personal judgement, not a summary of a trillion documents; and (3) that summary of those trillion documents has to be unique, but the question has no "right answer". Whilst computer science has the driving seat over what "intelligence" is, we will forever be stuck with this incredibly diminished view of our own capacities and the size of the technological challenge. The goal isnt to sift through everything we have already done and "take a mean", the goal is to produce a system which could have done "everything we have done" without us.
- gowld 5y ago
  Experiments are data collection. It's the iinput to the thinking process.
- bbqbbqbbq 5y ago
  This is why we work with our tools though, no? A NN can decide where the ambiguities lie in the data set and communicate that to the data scientists who are working with it, who can then provide data to help disambiguate the data.
- shawnz 5y ago
  That clearly doesn't explain how you get from the biological system to intelligence though. NNs can be imagined as a (highly simplistic) form of repeated experiment too.
  - mjburgess 5y ago
    Intelligence starts when the internal physical structure of a system is "dynamically reflective" of its external environment in a way which is stable over time (basically, "complex hysteresis"). You get this with a hard drive (+CPU, etc.) sure. I'd call this sort of minimal intelligence merely "reactive". You get, let's say "adaptive intelligence" when the physiological structure being changed adapts the system so that it is able to interact with its environment more (eg., it can move differently). To get to more advanced forms we need an explicit reasoning process which can represent this physical state to the system itself to engage in inference. Let's say "cognitive intelligence". We get typical mammalian intelligence when the broader physiological structure (in particular the sensory-motor structure) of the system is actually guided by this explicit reasoning process. (Eg., a cyclist grows their muscles differently by reasoning-when-cycling). Let's call this "skill intelligence". You get human intelligence when the explicit reasoning process becomes communicable, ie., when interior representations can be shared without the physical activity of acquiring those representations. Human intelligence is really "outside-in" in a very important way, which AI today also neglects -- it took 100bn dead apes to write a book, not one. Let's call this "socio-symbolic intelligence". What we have today is really just systems of "reactive intelligence" with weird frankesianian organs attached to them "look at alex turn the lights off!!!!". Alexa isnt turning the lights off in the manner of the "socio-symbolc intelligence" we attribute to alexa naively (and delusionally!). Alexa is a reactive system which has something of socio-symbolic significance (to us, not it!) glued on. Alexa does not intend to turn the lights off, and we're not communicating our intent to her. She's a harddrive with a SATA cable to a lightswitch.
    - shawnz 5y ago
      It seems to me like a "reactive intelligence" is basically equivalent to an "adaptive intelligence" which just hasn't begun making use of all of its possible outputs yet. Obviously even though we adapt to our environment, there are still ultimate limits to how far we can adapt.

by shawnz 5y ago

mjburgess 5y ago

direct, theory-laden, causal contact and pro-active engagement with their environments... eg., taking the stick out of the water. "data" (even from an animal's pov) is basically, by nature, ambiguous. Measurement is an event in the world (eg., light hitting the eye) which isn't somehow unambiguously informative. To over come this problem, basically, animals move stuff. The adaptable motor cortex of the most intelligent animals, therefore, isnt something to be tacked-on to "intelligence", it's the precondition for it. Glibly: tools before thoughts. Reason needs content to operate on, the content of our thoughts is built via our (sensory-)motor systems. The idea that we need ever more theoretically powerful models of reasoning here is a misdirection -- it misses that the heart of everything we know arrives via some form of repeated experiment. Every more automated "pure reasoning" either via statistics or symbolically, is always just a means of juicing the data we provide the system. Useful as a technology, but not as a genuine learning system. It will never have the means of resolving the many ambiguities within data itself. In the case, for example, of NLP -- the structure of 1 trillion documents will never enable a machine to answer the question "what do you like about what i'm wearing?" -- because (1) the machine isnt here with me; (2) i am asking for its personal judgement, not a summary of a trillion documents; and (3) that summary of those trillion documents has to be unique, but the question has no "right answer". Whilst computer science has the driving seat over what "intelligence" is, we will forever be stuck with this incredibly diminished view of our own capacities and the size of the technological challenge. The goal isnt to sift through everything we have already done and "take a mean", the goal is to produce a system which could have done "everything we have done" without us.

gowld 5y ago
Experiments are data collection. It's the iinput to the thinking process.
bbqbbqbbq 5y ago
This is why we work with our tools though, no? A NN can decide where the ambiguities lie in the data set and communicate that to the data scientists who are working with it, who can then provide data to help disambiguate the data.
shawnz 5y ago
That clearly doesn't explain how you get from the biological system to intelligence though. NNs can be imagined as a (highly simplistic) form of repeated experiment too.
- mjburgess 5y ago
  Intelligence starts when the internal physical structure of a system is "dynamically reflective" of its external environment in a way which is stable over time (basically, "complex hysteresis"). You get this with a hard drive (+CPU, etc.) sure. I'd call this sort of minimal intelligence merely "reactive". You get, let's say "adaptive intelligence" when the physiological structure being changed adapts the system so that it is able to interact with its environment more (eg., it can move differently). To get to more advanced forms we need an explicit reasoning process which can represent this physical state to the system itself to engage in inference. Let's say "cognitive intelligence". We get typical mammalian intelligence when the broader physiological structure (in particular the sensory-motor structure) of the system is actually guided by this explicit reasoning process. (Eg., a cyclist grows their muscles differently by reasoning-when-cycling). Let's call this "skill intelligence". You get human intelligence when the explicit reasoning process becomes communicable, ie., when interior representations can be shared without the physical activity of acquiring those representations. Human intelligence is really "outside-in" in a very important way, which AI today also neglects -- it took 100bn dead apes to write a book, not one. Let's call this "socio-symbolic intelligence". What we have today is really just systems of "reactive intelligence" with weird frankesianian organs attached to them "look at alex turn the lights off!!!!". Alexa isnt turning the lights off in the manner of the "socio-symbolc intelligence" we attribute to alexa naively (and delusionally!). Alexa is a reactive system which has something of socio-symbolic significance (to us, not it!) glued on. Alexa does not intend to turn the lights off, and we're not communicating our intent to her. She's a harddrive with a SATA cable to a lightswitch.
  - shawnz 5y ago
    It seems to me like a "reactive intelligence" is basically equivalent to an "adaptive intelligence" which just hasn't begun making use of all of its possible outputs yet. Obviously even though we adapt to our environment, there are still ultimate limits to how far we can adapt.