4 ms·
The sparse tensor perspective intrigues me, could you perhaps provide me with some references?
by joycian 7y ago
The sparse tensor perspective intrigues me, could you perhaps provide me with some references?
- GistNoesis 7y agoNot specifically. You can look at the literature for ontology, relation networks, semantic databases, reasoning. The video that made me clicked, was https://www.ted.com/talks/ajit_narayanan_a_word_game_to_communicate_in_any_language https://www.ted.com/talks/ajit_narayanan_a_word_game_to_comm... It depicts in a game to help autistic children better communicate, a simple order-independent language representation, which is isomorphic to graphs. If you do the math, you see that some triplets query like (queen, subclass, ?) is just a point-wise multiplication with the knowledge database sparse tensor in this sparse boolean tensor representation (the ? is a column of 1). Then it's just chaining requests. A graph of triplet is isomorphic to a boolean tensor of dimension (nbSubjects, nbRelations, nbObjects).
- mlevental 7y agothat person's description is unnecessarily jargony. sparse tensor just means vector that's mostly zero and has some ones. take your corpus of concepts and number them 0 to n-1. then each word you want to turn into a vec turns into a vector that has ones for concepts it's associated with and 0 elsewhere. learning that association is what word2vec does.
- rytill 7y agoCan you explain attention in this same way?
- mlevental 7y agonah i don't know much about LSTM but this looks like a fine tutorial https://towardsdatascience.com/intuitive-understanding-of-attention-mechanism-in-deep-learning-6c9482aecf4f https://towardsdatascience.com/intuitive-understanding-of-at...