4 ms·

Exactly. Glad to see this. I do think knowledge graphs are important to AI assistants and agents though and someone needs to build a knowledge graph solution fo

by dbish 2y ago

Exactly. Glad to see this. I do think knowledge graphs are important to AI assistants and agents though and someone needs to build a knowledge graph solution for that space. The idea of actual entities and relationships defined like triples with some schema and appropriately resolved and linked can be useful for querying and building up the right context. It may even be time to start bringing back some ideas from the schema.org back the day to standardize across agents/assistants what entities and actions are represented in data fed to them.

gaogao 2y ago
Yeah, one of the specific things I'd love to do is collaboratively bulking up WikiData more. It's missing a ton of low hanging fruit that people using an ML augmented tool could really make some good progress on, similar to ML assisted OpenStreetMapping work
- chottocharaii 2y ago
  This isn't going to work so long as WikiData is controlled by admins that misapply the notability criteria to delete information they subjectively consider trivial I think the next frontier is a a wiki-style, collaborative site, deliberately purposed at storing information for LLMs
  - andai 2y ago
    Is it bad that my first thought was "that's going to be a huge amount of work and end up with a lot of gaps and blind spots... we should have GPT-4 polulate it!" I think it could work if each entry has to be peer reviewed by 3 (more?) humans. Although, AIs are better than humans at captchas now, so I'm not exactly sure how that would work either way...
TrueDuality 2y ago
Yeah precisely. Knowledge graphs are simple to think about but as soon as you look into them you realize all the complexity is in the creation of a meaningful ontology and loading data into that ontology. I actually think LLMs can be massively useful for building up the ontology but probably not in the creation of the ontology itself (far too ambiguous and large/conceptual task for them right now).
- kiminemism 2y ago
  How do we build ontology using LLMs? Will the building blocks be like the different parts of a brain? P.S I am assuming that by "creation of ontology itself" means creation of AGI.
  - dbish 2y ago
    Ontologies are just defining what certain category, words, and entity types mean. Commonly used in NLP for data representation (“facts”/triples/etc.) in knowledge graphs and other places where the definition of an ontology helps provide structure. This doesn’t have anything to do with AGI or brains. They are typically created or tuned by humans and then models fit/match/resolve entities to match the ontology.
  - deleted 2y ago
    [deleted]
  - TrueDuality 2y ago
    @dbish nailed it, but I can give you a bit more concrete example. Continuing off the light example I started off with. An ontology for knowing what city a person currently is in. We have two classes of entities, a person, and a city. There is a single relationship type "LocatedAt" you can add and remove edges to indicate where a person is and you can construct some verification rules such as "a person can only be in one city at time". To have an LLM construct a knowledge graph of where someone is (and I know this example is incredibly privacy invasive but its a simple concrete example not representative). Imagine giving an LLM access to all of your text messages. You can imagine giving it a prompt along the lines of "identify who is being discussed in this series of messages, if someone indicates where they are physically located report that as well" (you'd want to try harder than that, keeping it simple). You could get an output that says something like `{"John Adams": "Philadelphia, PA, US"}`. If either the left or right side are missing create them. Then remove any LocatedAt edges for the left side and add one between these two entities. You have a simple knowledge graph. Seems easy enough, but try to ask slightly harder questions... When did we know John Adams was in Philadelpha? Have they been there before? Where were they before Philadelphia? The ontology I just developed isn't capable of representing that data. You can of course solve these problems and there are common ontological patterns for representing it. The point is, you kind of need to know the kind of questions you want to ask about your data when you're building your ontology and you're always going to miss something. Usually you find out the unknown questions you want to ask of the data only after you've already built your system and started asking it questions. It's the follow-ups that kill you. There has been a lot of work on totally unstructured ontologies as well, but you're moving the hard problem elsewhere not solving it. Instead of having high quality data you can't answer every question with, you have arbitrary associations that may mean the same thing and thus any query you make is likely _missing_ relevant data and thus inaccurate. Huge headache to go down, but honestly I think it is a worthwhile one. Previously if you changed your ontology to answer a new question, a human would have to go through and manually and painstakingly update your data to the new system. This is boring, tedious, easy-to-get-wrong-due-to-inattention kind of work. It's not complex, its not hard, its very easy to double check but it does require an understanding of language. LLMs are VERY capable of doing this kind of work, and likely more accurately.

by dbish 2y ago

gaogao 2y ago

Yeah, one of the specific things I'd love to do is collaboratively bulking up WikiData more. It's missing a ton of low hanging fruit that people using an ML augmented tool could really make some good progress on, similar to ML assisted OpenStreetMapping work

chottocharaii 2y ago
This isn't going to work so long as WikiData is controlled by admins that misapply the notability criteria to delete information they subjectively consider trivial I think the next frontier is a a wiki-style, collaborative site, deliberately purposed at storing information for LLMs
- andai 2y ago
  Is it bad that my first thought was "that's going to be a huge amount of work and end up with a lot of gaps and blind spots... we should have GPT-4 polulate it!" I think it could work if each entry has to be peer reviewed by 3 (more?) humans. Although, AIs are better than humans at captchas now, so I'm not exactly sure how that would work either way...

TrueDuality 2y ago

Yeah precisely. Knowledge graphs are simple to think about but as soon as you look into them you realize all the complexity is in the creation of a meaningful ontology and loading data into that ontology. I actually think LLMs can be massively useful for building up the ontology but probably not in the creation of the ontology itself (far too ambiguous and large/conceptual task for them right now).

kiminemism 2y ago
How do we build ontology using LLMs? Will the building blocks be like the different parts of a brain? P.S I am assuming that by "creation of ontology itself" means creation of AGI.
- dbish 2y ago
  Ontologies are just defining what certain category, words, and entity types mean. Commonly used in NLP for data representation (“facts”/triples/etc.) in knowledge graphs and other places where the definition of an ontology helps provide structure. This doesn’t have anything to do with AGI or brains. They are typically created or tuned by humans and then models fit/match/resolve entities to match the ontology.
- deleted 2y ago
  [deleted]
- TrueDuality 2y ago
  @dbish nailed it, but I can give you a bit more concrete example. Continuing off the light example I started off with. An ontology for knowing what city a person currently is in. We have two classes of entities, a person, and a city. There is a single relationship type "LocatedAt" you can add and remove edges to indicate where a person is and you can construct some verification rules such as "a person can only be in one city at time". To have an LLM construct a knowledge graph of where someone is (and I know this example is incredibly privacy invasive but its a simple concrete example not representative). Imagine giving an LLM access to all of your text messages. You can imagine giving it a prompt along the lines of "identify who is being discussed in this series of messages, if someone indicates where they are physically located report that as well" (you'd want to try harder than that, keeping it simple). You could get an output that says something like `{"John Adams": "Philadelphia, PA, US"}`. If either the left or right side are missing create them. Then remove any LocatedAt edges for the left side and add one between these two entities. You have a simple knowledge graph. Seems easy enough, but try to ask slightly harder questions... When did we know John Adams was in Philadelpha? Have they been there before? Where were they before Philadelphia? The ontology I just developed isn't capable of representing that data. You can of course solve these problems and there are common ontological patterns for representing it. The point is, you kind of need to know the kind of questions you want to ask about your data when you're building your ontology and you're always going to miss something. Usually you find out the unknown questions you want to ask of the data only after you've already built your system and started asking it questions. It's the follow-ups that kill you. There has been a lot of work on totally unstructured ontologies as well, but you're moving the hard problem elsewhere not solving it. Instead of having high quality data you can't answer every question with, you have arbitrary associations that may mean the same thing and thus any query you make is likely _missing_ relevant data and thus inaccurate. Huge headache to go down, but honestly I think it is a worthwhile one. Previously if you changed your ontology to answer a new question, a human would have to go through and manually and painstakingly update your data to the new system. This is boring, tedious, easy-to-get-wrong-due-to-inattention kind of work. It's not complex, its not hard, its very easy to double check but it does require an understanding of language. LLMs are VERY capable of doing this kind of work, and likely more accurately.