6 ms·
Context Is Everything: Finding Meaning Statistically in Semantic Spaces
- sizzle 8y agoVery cool. Any interesting uses for this? I want to develop a 'cognitive debiasing' ML system. It will take real time spoken languages and parse it syntactically to output a dibiased version of the input. Let me know if it have any resources or insight on this. Thanks!
- 3131s 8y agoBy "debiasing" do you mean word sense disambiguation?
- sizzle 8y agoNot sure what the technical term is but by debias I meant the ability to detect cognitive biases [1] from real time spoken input and remove them from the output. The goal would be to help humans understand their biased thinking and over time make us less biased in our decision making process. [1] https://en.m.wikipedia.org/wiki/List_of_cognitive_biases https://en.m.wikipedia.org/wiki/List_of_cognitive_biases
- gradys 8y agoIt sounds like you want to be able to generate language with the same basic meaning as the original but without whatever bias was there. I'd recommend starting by trying to detect the presence of the bias first. Language generation is a very difficult problem in general and this is an especially difficult instance of it. Even detecting bias will be challenging because of how subtle a phenomenon it is, but it's much more tractable than rewriting what was said to not include the detected bias.
- Radim 8y agoThe article seems a little rough (draft?), but the money shot is the algo in Section 5. Looks pretty neat and straightforward! TL;DR: 1. Build a special distance(word1, word2) metric using a set of word vectors trained elsewhere (such as GloVe). This distance works better than cosine distance. 2. Given a document (a sentence, a paragraph… basically, a sequence of words), calculate the "importance" of each word as a sigmoid over distances(word, avg(all_words)). 3. To embed a document, simply do a weighted average of its word vectors, where the weight of each word equals the importance above.
- ContextArxiv 8y agoHi, author here! I can confirm everything you said. A conference version is in the works, though this one is essentially complete
- braindead_in 8y agoDo you plan to release an implementation? I would definitely like to play around with it.
- ContextArxiv 8y agoAbsolutely. The current priority is conference submission, but releasing a simple interface is definitely the plan, as appropriate
- ContextArxiv 8y agohttps://github.com/ezelikman/Context-Is-Everything https://github.com/ezelikman/Context-Is-Everything The code will be posted here when available!
- auvrw 8y agohere's an impl of something like (but actually probably nothing like) 1. in the above tl;dr (disclaimer: i dr) https://github.com/ransomw/botiful-soup/blob/master/mibot/src/clj/mibot/learn/metrics.clj#L17 https://github.com/ransomw/botiful-soup/blob/master/mibot/sr... as i recall, there's a lot of neato REPL-y graphvis to try in https://github.com/ransomw/botiful-soup/tree/master/mibot/dev https://github.com/ransomw/botiful-soup/tree/master/mibot/de... ---- while i'm here, let me mention: Gärdenfors books like "Geometry of Meaning" are hearty food for thought about natural language semantics