3 ms·

Interesting to compare this semantic map with the semantic maps that Mikolov's Word2vec processes produce and Word2vec's resultant semantic vector math (King -

by deepnet 10y ago

Interesting to compare this semantic map with the semantic maps that Mikolov's Word2vec processes produce and Word2vec's resultant semantic vector math (King - Man + Woman ~= Queen). Perhaps Hinton's Vectors of Thought do have a wetware analogy.

igravious 10y ago
What would this mean if so? And if not, what would that imply? :)
- deepnet 10y ago
  In terms of (Artificial) Neural Networks it implies their internal representation is functional not merely symbolic. Soumith has shown this applies to visual modalities and smile vectors or wearing sunglasses vectors are similarly present[1]. This types of vector algebra has some use in translating between languages and modalities. Karpathy[2] has demonstrated an internal vector of a descriptive sentence can 'plugged into' an image recognising network and used to find vectors representing pictures. Perhaps a sentence vector could even generate a picture. What Neural Nets do internally is still mysterious. (A Highly speculative example) If DeepGo's internal vectors could translate to the modality of plain English we might glimpse its 'mind' at work. What this means for brains is anyones guess, I find inspiration in Geoff Hinton's guesses[3]. [1] - searching [2] https://www.youtube.com/watch?v=ZkY7fAoaNcg https://www.youtube.com/watch?v=ZkY7fAoaNcg [3] http://www.computing.co.uk/ctg/news/2409871/document-search-engines-will-be-able-to-think-and-reason-like-people-argues-ai-expert http://www.computing.co.uk/ctg/news/2409871/document-search-...
  - jawarner 10y ago
    The brain evolved, and neural nets are trained, to optimize for high-density, "low-cost" information storage. So it would make sense if there is a structural correlation between the two in terms of how information is represented. I suspect that more biologically realistic neural nets would lead to more correlation.
  - deepnet 10y ago
    [1] - Radford, Metz, Chintala https://github.com/Newmu/dcgan_code#arithmetic-on-faces https://github.com/Newmu/dcgan_code#arithmetic-on-faces
  - igravious 10y ago
    Your "[1] - searching" made me smile :) Gonna read and watch your links and do some thinking of my own, thanks.

by deepnet 10y ago

igravious 10y ago

What would this mean if so? And if not, what would that imply? :)

deepnet 10y ago
In terms of (Artificial) Neural Networks it implies their internal representation is functional not merely symbolic. Soumith has shown this applies to visual modalities and smile vectors or wearing sunglasses vectors are similarly present[1]. This types of vector algebra has some use in translating between languages and modalities. Karpathy[2] has demonstrated an internal vector of a descriptive sentence can 'plugged into' an image recognising network and used to find vectors representing pictures. Perhaps a sentence vector could even generate a picture. What Neural Nets do internally is still mysterious. (A Highly speculative example) If DeepGo's internal vectors could translate to the modality of plain English we might glimpse its 'mind' at work. What this means for brains is anyones guess, I find inspiration in Geoff Hinton's guesses[3]. [1] - searching [2] https://www.youtube.com/watch?v=ZkY7fAoaNcg https://www.youtube.com/watch?v=ZkY7fAoaNcg [3] http://www.computing.co.uk/ctg/news/2409871/document-search-engines-will-be-able-to-think-and-reason-like-people-argues-ai-expert http://www.computing.co.uk/ctg/news/2409871/document-search-...
- jawarner 10y ago
  The brain evolved, and neural nets are trained, to optimize for high-density, "low-cost" information storage. So it would make sense if there is a structural correlation between the two in terms of how information is represented. I suspect that more biologically realistic neural nets would lead to more correlation.
- deepnet 10y ago
  [1] - Radford, Metz, Chintala https://github.com/Newmu/dcgan_code#arithmetic-on-faces https://github.com/Newmu/dcgan_code#arithmetic-on-faces
- igravious 10y ago
  Your "[1] - searching" made me smile :) Gonna read and watch your links and do some thinking of my own, thanks.