4 ms·

> Sure, but all this is only within one space. The "internal language" is a formalism of that space, and not beyond. This is incorrect: You can swap encoders

> Sure, but all this is only within one space. The "internal language" is a formalism of that space, and not beyond. This is incorrect: You can swap encoders and final fully connected sections, then mildly retrain the network to substantially save on training effort — “transfer learning”. Further, you can compose networks, eg image recognition on top of something extracting a structural map. This implies that the structures they’re “learning” generalize to different contexts — with a bit of retraining. In much the way your knowledge of programming transfers between languages (swapping encoders) and tasks (swapping final connected network). Your point seems deeply based on this incorrect understanding of DNNs.

mjburgess 5y ago
Sure, as set up by the way we have constructed the dataset. "Green" isn't near "Leaves" because the world placed them there. All these properties follow from properties of the data which we have structured to produce these solutions. We arrange the target domain of one dataset up, so that the target domain of another is aligned --- and in doing so, solve the only problem which requires intelligence. Yes, the character-space structure of Y labelled stuff aligns with the pixel space structure of Y labelled stuff. And feature space templates can likewise be associated. This is just more of the same. King +Woman isn't Queen --- it isn't anything. This isn't reasoning. And likewise PicOfKing+PicOfW isnt PicOfQueen. This is schizophrenic pseudoscience --- it's actually the mind of a schizophrenic who obtains sequential thoughts just by non-semantic associations. The latent structure being learnt in both cases is just the coincidences we rig in the Y domain --- Inhave no doubt they 'transfer', we wrote them in
- zmgsabst 5y ago
  We don’t write them in though. You can show a DNN a large sample of photos of animals, retain the lower sections of the network, and then show it a large sample of landscapes — and it will be able to recognize those faster precisely because a well-trained network picked up on the latent structure of shapes, and can reason about shapes in its internal language for describing pictures. It’s acquiring genuine semantic information — and learning about new experiences relative to that previous acquired understanding. We didn’t create that similarity of form in nature — and DNNs discover that in much the way we do: finding recurring, abstract patterns to utilize in our internal language.
  - mjburgess 5y ago
    We labelled them -- their pixel space geometric structure has no inherent relevance to their character space structure --- and the only inferential contexts this will work in is where we rig the deployment to only require these sorts of coincidences. Animals acquire robust techniques which are not brittle to these sorts of thin contexts. We do so by not templating pregiven data -- but by coordinating our bodies in the world, and rearranging it, so as to determine how to regulate ourselves in response to it. Only a teen reading cosmo would think King-Man=Woman. The reason we can even interpret this thin formalism is because of our robust understanding of the concepts that these character symbols name. With these concepts we don't cross domains, we regulate our own imagination, and so on, so as to create representations which are thick across an infinity of domains. Talking in terms of feature domains has it backwards -- it assumes they're available for measurement. We don't transfer thin templates across domains. Our representations aren't within a feature space --- they're techniques for self-regulation. With these we can construct templates in an infinite number of different feature spaces -- ie., project by imagination from our techniques
- mistrial9 5y ago
  the downvotes on this one show some blind alligiance to word2vec it seems to me; I tend towards questioning the playing field, not clamping to a (large) set of useful vactors to be the new "hammer looks for nails"

mjburgess 5y ago

Sure, as set up by the way we have constructed the dataset. "Green" isn't near "Leaves" because the world placed them there. All these properties follow from properties of the data which we have structured to produce these solutions. We arrange the target domain of one dataset up, so that the target domain of another is aligned --- and in doing so, solve the only problem which requires intelligence. Yes, the character-space structure of Y labelled stuff aligns with the pixel space structure of Y labelled stuff. And feature space templates can likewise be associated. This is just more of the same. King +Woman isn't Queen --- it isn't anything. This isn't reasoning. And likewise PicOfKing+PicOfW isnt PicOfQueen. This is schizophrenic pseudoscience --- it's actually the mind of a schizophrenic who obtains sequential thoughts just by non-semantic associations. The latent structure being learnt in both cases is just the coincidences we rig in the Y domain --- Inhave no doubt they 'transfer', we wrote them in

zmgsabst 5y ago
We don’t write them in though. You can show a DNN a large sample of photos of animals, retain the lower sections of the network, and then show it a large sample of landscapes — and it will be able to recognize those faster precisely because a well-trained network picked up on the latent structure of shapes, and can reason about shapes in its internal language for describing pictures. It’s acquiring genuine semantic information — and learning about new experiences relative to that previous acquired understanding. We didn’t create that similarity of form in nature — and DNNs discover that in much the way we do: finding recurring, abstract patterns to utilize in our internal language.
- mjburgess 5y ago
  We labelled them -- their pixel space geometric structure has no inherent relevance to their character space structure --- and the only inferential contexts this will work in is where we rig the deployment to only require these sorts of coincidences. Animals acquire robust techniques which are not brittle to these sorts of thin contexts. We do so by not templating pregiven data -- but by coordinating our bodies in the world, and rearranging it, so as to determine how to regulate ourselves in response to it. Only a teen reading cosmo would think King-Man=Woman. The reason we can even interpret this thin formalism is because of our robust understanding of the concepts that these character symbols name. With these concepts we don't cross domains, we regulate our own imagination, and so on, so as to create representations which are thick across an infinity of domains. Talking in terms of feature domains has it backwards -- it assumes they're available for measurement. We don't transfer thin templates across domains. Our representations aren't within a feature space --- they're techniques for self-regulation. With these we can construct templates in an infinite number of different feature spaces -- ie., project by imagination from our techniques
mistrial9 5y ago
the downvotes on this one show some blind alligiance to word2vec it seems to me; I tend towards questioning the playing field, not clamping to a (large) set of useful vactors to be the new "hammer looks for nails"