5 ms·
The graph algorithm they're describing is basically Manaal Faruqui's "retrofitting", although they don't cite him. I will make the charitable assumption that th
by rspeer 10y ago
The graph algorithm they're describing is basically Manaal Faruqui's "retrofitting", although they don't cite him. I will make the charitable assumption that they came up with it independently (and managed to miss the excitement about it at NAACL 2015).
Here's why not to be sad: Retrofitting and its variants are quite effective, and surprisingly, they're not really that computationally intensive. I use an extension of retrofitting to build ConceptNet Numberbatch [1], which is built from open data and is the best-performing semantic model on many word-relatedness benchmarks.
The entirety of ConceptNet, including Numberbatch, builds in 5 hours on a desktop computer.
Big companies have resources, but they also have inertia. Some problems are solved by throwing all the resources of Google at the problem, but that doesn't mean it's the only way to solve the problem.
[1] https://github.com/commonsense/conceptnet-numberbatch https://github.com/commonsense/conceptnet-numberbatch
- igravious 10y agoI can't find a single comment thread on HN where ConceptNet is discussed. Bummer. See here: https://hn.algolia.com/?query=conceptnet&sort=byPopularity&prefix&page=0&dateRange=all&type=story https://hn.algolia.com/?query=conceptnet&sort=byPopularity&p... Short of signing up to the mailing list on Google Groups is there any info out there on the net that tells me: a) How ConceptNet relates to efforts like Cyc's OpenCyc and the big G's Knowledge Graph? b) How to add domain specific concepts, attributes, and relations to ConceptNet Both these things are the first that pop into my mind but the website: http://conceptnet5.media.mit.edu/ http://conceptnet5.media.mit.edu/ doesn't seem to cover them.
- rspeer 10y agoHm, I know I've brought it up before. In relation to OpenCyc: ConceptNet doesn't have CycL or any sort of logical predicates -- the assumption is that you are just doing fuzzy machine learning over ConceptNet. I'm pretty sure you need the rest of Cyc to do anything interesting with CycL, though. ConceptNet takes advantage of linked open data when it can, so it does contain attempted translations of the facts in OpenCyc. Compared to the Google Knowledge Graph: well, you can't really use the Knowledge Graph without being at Google, or making a research agreement with them, right? But from what I've seen of it, and of its predecessor Freebase, I think it focuses a lot on named entities: particularly things you can look up on Wikipedia or things you can buy. Which is fine information. I'd say it's a different segment of linked data than "what words mean", which is ConceptNet's focus. So maybe think of it as more like a bigger WordNet than a smaller Knowledge Graph. How to add domain-specific concepts to ConceptNet: the unsatisfying answer is that you can get the code for ConceptNet and alter the build process to include new data sources (try it on the 5.5 branch if you attempt this). And the startuppy sellout answer is that doing this automatically is one of the things my company Luminoso is for (http://www.luminoso.com http://www.luminoso.com). Thanks for asking, and I'll try to clarify this kind of stuff when I deploy the new site.
- igravious 10y agoThanks for the response. I'll keep an eye on your progress, dig deeper into what you already do, and have a play with your tools.
- jerad 10y agoWow, there's something funny going on with HN search there. There's a quite a few discussions about ConceptNet. https://www.google.com/search?q=site%3Ahttps%3A%2F%2Fnews.ycombinator.com%2F%20conceptnet https://www.google.com/search?q=site%3Ahttps%3A%2F%2Fnews.yc...
- igravious 10y agoIt differs if you search by Stories or Comments. There are hits if you search by Stories but none of the hits contain posts which is what threw me.
- exgrv 10y agoThe graph algorithm described in the blogpost is more related to label propagation (which is more than 10 years old), than to "retrofitting". And the Google paper linked in the blogpost is citing the relevant literature correctly.
- rspeer 10y agoI probably sounded more accusatory than I should have, and I apologize for that wording. But I do think this is much more like retrofitting than like label propagation. It's the vectors that are being propagated, as I understand it, not labels.
- amelius 10y ago> Big companies have [...] inertia. This used to be the case. But companies nowadays are often more structured like clusters of small startups, especially for innovative products and moonshots and such.