3 ms·
Do people generally read these things before upvoting? Legitimately curious. It lists some possible goals to achieve more general human-like intelligence beyon
by andreyk 11y ago
Do people generally read these things before upvoting? Legitimately curious.
It lists some possible goals to achieve more general human-like intelligence beyond the fancy function approximation we get with deep supervised learning, as stated in the abstract for both architectural and functional perspectives. In general I find the language fairly wishi-washy and the writing often awkward, but it is a nice summary of relevant thoughts and concepts. Beyond the abstract, here is a bit of summary and my thoughts.
For architectural aspects, it lists:
1) Unsupervised - Agrees with LeCun, Bengio, etc. But not sure it's fair to conclude this yet, maybe it should be reinforcement? our brains are prewired to do some things
2) Compositional - basically hierarchical, aka deep. Seems reasonable.
3) Sparse and Distributed - again plausible and empirically seen in deep learning. One reason ReLu neurons are nice is that they lead to sparser distributed representations.
4) Objectiveless - a metaphysical statement having to do with the Chinese room argument? This seems to mean not optimizing an objective function with gradient descent, and instead "Clearly, the learning algorithm should have a goal, which might be defined very broadly such as the theory of curiosity, creativity and beauty described by J. Schmidhuber". Seems vague and not clear.
5) Scalable - Again not the best choice of words, it seems to argue for parallelism as well as a "hierarchical structure allowing for separate parallel local and global updates of synapses, scalability and unsupervised learning at the lower levels with more goal-oriented finne-tuning in higher regions. " I am disappointed no discussion of memristors or neuromorphic computing was here.
For function aspects, it lists:
1) Compression - sure, pattern matching is in a sense compression so this seems fairly obvious.
2) Prediction - "Whereas the smoothness prior may be considered as a type of spatial coherence,
the assumption that the world is mostly predictable corresponds to temporal or
more generally spatiotemporal coherence. This is probably the most important ingredient of a general-purpose learning procedure." Again, reasonable enough.
3) Understanding - basically equivalent to predicting?
4) Sensorimotor - not clear? Similar to human eye movement?
5) Spatiotemporal Invariance - "one needs to
inject additional contex" having constant concepts of things?
6) Context update/pattern completion - "The last functional component postulated by this paper is a continuous (in the-
ory) loop between bottom-up predictions and top-down context." Constant cycling between prediction and word state update, pretty clear.
- deleted 11y ago[deleted]
- mindcrime 11y agoDo people generally read these things before upvoting? Legitimately curious. I don't. And that's because HN doesn't have separate "upvote" and "save" features... upvoting is saving (or, saving is upvoting, however you want to look at it). So I save (upvote) anything that meets the bar of "the title is interesting enough for me to think I might want to read this eventually".
- andreyk 11y agoA save feature would be nice perhaps. Although maybe the philosophy of HN is precisely "upvote if something seems interesting enough to read" rather than (or in addition to) "upvote if you have read this and think it's good".
- bcook 11y agoI usually upvote the comments section of a post, rather than the linked URL's information.
- graycat 11y agoThanks for the summary. Net, I was afraid of some such. I.e., IMHO, it won't even start to work. Here's my first-cut architecture: The core of the intelligence is some concepts and relationships between them. The relationships are based on historical input data from experience since birth. The concepts are close to nouns in language. The relationships are close but not as close to verbs in language. The relationships are closely related to what we regard as causes. The relationships can be moderated or qualified by, right, adverbs. And, sure, adjectives can modify the nouns. Then, start like a baby does, with just nouns and then some simple verbs with sentences of just two words. As for say, kitty cats, early on the input is not in words or language at all -- actual language as a way to get input data, say, fairly directly to concepts with nouns, is later. For rational and deductive reasoning, that is just a special case where the causal relationships are taken much more seriously than usual. For vision, to do at all well, need to do at least fairly well with concepts -- that is, vision needs to identify objects for which the intelligence already has, or at least is in the process of learning, concepts and, say, recognizing a 3D object from just its 2D image. I'm busy with my startup now, but later maybe I'll try to code up what I just described. Here I'd be using electronic logic; how biologic logic does such things, e.g., search the collection of 3D internal visualizations of concepts that are 3D objects, I don't have a clue.