5 ms·

Is anyone using this? It's from 2019.

cess11 2y ago
If someone is it's probably pretty niche. RDF is a bit simpler and still not commonly used due to its perceived complexity compared to RDBMS tables or JSON.
- nl 2y ago
  This completely misunderstands what this is. In this context "knowledge graph" means a representation of knowledge independent of it's serialization format (ie, RDF vs whatever). It's usable wherever you have a key value (key:value of relationship) or a triple (key:relationship:something). This is probably 85% of everything that is stored in a database, with the exception being paths that vary enough that you can't use the path as a key or relationship. In particular, most trees are suitable if you use collapse the path down to a single relationship. I've done a bunch of work on graph embedding. They are very effective for use in anything that can be thought of as a recommendation system ("this person would like these books") or similarity ("this person is similar to these people"). Back when I was working on them I found Starspace wonderfully easy and effective: https://github.com/facebookresearch/StarSpace https://github.com/facebookresearch/StarSpace
  - cess11 2y ago
    I didn't compare RDF directly to the technique in TFA.
  - DrDroop 2y ago
    It is wrong the call this a "complete misunderstanding" as the paper clearly deals with triplets and formulates a different encoding scheme for entities and relations. Ok, so it is not RDF or even hyper graphs but it is dealing with a graph structure.
    - nl 2y ago
      Sure, but as I explain almost everything can fit that structure. The misunderstanding is the " probably pretty niche" part since related techniques (graph embedding generally) are so widely used.
VHRanger 2y ago
These exotic (non linear algebra based) embedding representations are often slow to take off unless they have an obvious use case. The other one that I've always been curious of is Poincarré Embeddings [1] - where the embedding also has a hierarchical representation of the space. There's issues with these becoming popular: 1. Querying the embeddings requires more math knowledge than just "lol cosine similarity". This also requires to write code for the query 2. You can often easily match the performance with regular embeddings by just adding dimensions and training more. So the advantage of exotic embeddings has to be with the information in the more complex mathematical abstraction. So they need a killer usecase to become popular, it's hard to move the needle. [1]: https://arxiv.org/abs/1705.08039 https://arxiv.org/abs/1705.08039
- nico 2y ago
  Super interesting Do you think it makes sense to have a group of models, each with more ad-hoc embeddings, and coordinate them to respond according to the domain of the input? Do multi-modal models use the same embedding type/structure for an image, sound, text?
  - VHRanger 2y ago
    I think your first question is open for people to explore. The answer to the second is yes - it's all vector embeddings, and they're aligned to each other by finding a dataset that matches pairs (eg. images with captions) The real use for exotic embeddings will have to be in analyzing the embeddings themselves I think, otherwise it's easier to shove normal vectors downstream into other models.
adultSwim 2y ago
The paper was cited over 500 times, https://scholar.google.com/scholar?cites=15051838971302171507&as_sdt=5,33&sciodt=0,33&hl=en https://scholar.google.com/scholar?cites=1505183897130217150...
adultSwim 2y ago
QuatE is implemented in PyKEEN, a library for KnowEdge graph EmbeddiNgs, https://github.com/pykeen/pykeen https://github.com/pykeen/pykeen

If someone is it's probably pretty niche. RDF is a bit simpler and still not commonly used due to its perceived complexity compared to RDBMS tables or JSON.

nl 2y ago
This completely misunderstands what this is. In this context "knowledge graph" means a representation of knowledge independent of it's serialization format (ie, RDF vs whatever). It's usable wherever you have a key value (key:value of relationship) or a triple (key:relationship:something). This is probably 85% of everything that is stored in a database, with the exception being paths that vary enough that you can't use the path as a key or relationship. In particular, most trees are suitable if you use collapse the path down to a single relationship. I've done a bunch of work on graph embedding. They are very effective for use in anything that can be thought of as a recommendation system ("this person would like these books") or similarity ("this person is similar to these people"). Back when I was working on them I found Starspace wonderfully easy and effective: https://github.com/facebookresearch/StarSpace https://github.com/facebookresearch/StarSpace
- cess11 2y ago
  I didn't compare RDF directly to the technique in TFA.
- DrDroop 2y ago
  It is wrong the call this a "complete misunderstanding" as the paper clearly deals with triplets and formulates a different encoding scheme for entities and relations. Ok, so it is not RDF or even hyper graphs but it is dealing with a graph structure.
  - nl 2y ago
    Sure, but as I explain almost everything can fit that structure. The misunderstanding is the " probably pretty niche" part since related techniques (graph embedding generally) are so widely used.

VHRanger 2y ago

These exotic (non linear algebra based) embedding representations are often slow to take off unless they have an obvious use case. The other one that I've always been curious of is Poincarré Embeddings [1] - where the embedding also has a hierarchical representation of the space. There's issues with these becoming popular: 1. Querying the embeddings requires more math knowledge than just "lol cosine similarity". This also requires to write code for the query 2. You can often easily match the performance with regular embeddings by just adding dimensions and training more. So the advantage of exotic embeddings has to be with the information in the more complex mathematical abstraction. So they need a killer usecase to become popular, it's hard to move the needle. [1]: https://arxiv.org/abs/1705.08039 https://arxiv.org/abs/1705.08039

nico 2y ago
Super interesting Do you think it makes sense to have a group of models, each with more ad-hoc embeddings, and coordinate them to respond according to the domain of the input? Do multi-modal models use the same embedding type/structure for an image, sound, text?
- VHRanger 2y ago
  I think your first question is open for people to explore. The answer to the second is yes - it's all vector embeddings, and they're aligned to each other by finding a dataset that matches pairs (eg. images with captions) The real use for exotic embeddings will have to be in analyzing the embeddings themselves I think, otherwise it's easier to shove normal vectors downstream into other models.

adultSwim 2y ago

The paper was cited over 500 times, https://scholar.google.com/scholar?cites=15051838971302171507&as_sdt=5,33&sciodt=0,33&hl=en https://scholar.google.com/scholar?cites=1505183897130217150...

adultSwim 2y ago

QuatE is implemented in PyKEEN, a library for KnowEdge graph EmbeddiNgs, https://github.com/pykeen/pykeen https://github.com/pykeen/pykeen