3 ms·
And in the reverse, Wikidata has a lot of gaps in its annotations, where human labelling could be augmented by LLMs. I wrote some stuff on both ground response
by gaogao 3y ago
And in the reverse, Wikidata has a lot of gaps in its annotations, where human labelling could be augmented by LLMs. I wrote some stuff on both ground response and adding more stuff to WikiData a while ago https://friend.computer/jekyll/update/2023/04/30/wikidata-llms.html https://friend.computer/jekyll/update/2023/04/30/wikidata-ll...
- jsemrau 3y agoWould be a good idea to create an annotation model like DALL-E 3 had done.
- unshavedyak 3y agoYup. This is my gut to where LLMs will really explode. Let them augment data just a bit, train on improved data, augment more, train again - etcetc. If we take things slow i suspect in the long run it'll be really beneficial to multiple paradigms. I know people say training bots on bot data is bad, but A: it's happening anyway, and B: it can't be worse than the actual garbage they get trained on in a lot of cases anyway.. can it?
- Filligree 3y agoPixart-alpha provides an example of C: The bot labels can be dramatically better than the human labels. Even though they used LLaVA, and LLaVA isn't all that good compared to gpt-4.
- gaogao 3y agoTraining on bot data can be bad when it's ungrounded and basically hallucinations on hallucinations. Having LLMs help curate something grounded is generally reasonable. Functionally, it's somewhat similar to how some training is using generated subtitles of videos for training video/text pairs; it's very feasible to also go and clean those up, even though it is bot data.
- ori_b 3y ago> Yup. This is my gut to where LLMs will really explode Yes, indeed. This is one place where LLMs can make it look like a bomb went off.
- prosqlinjector 3y agoWe can do polynomial regression of data sets that looks equally plausible, but it's not real data.
- foobarchu 3y ago> A: it's happening anyway This is never a valid defense for doing more of something.
- akjshdfkjhs 3y agoplease no! The cost of now having unknown false data in there would completely ruin the value of the whole effort. The entire value of the data (which is already everywhere anyway) is the "cost" contributors paid via heavy moderation. If you do not understand why that is diametrically opposite of adding/enriching/augmenting/whichever euphemism with LLMs, I don't know what else to say.
- cj 3y ago100%. I live in a hamlet of a larger town in the US, and was curious what the population of my hamlet is. There’s a Wikipedia page for the hamlet, but it’s empty. No population data, etc. I’d much rather see no data than a LLM’s best guess. I’m guessing a LLM using the data would also perform better without approximated or “probably right” information.
- gaogao 3y agoYeah, adding directly with an LLM is a bad idea. Instead, this would be basically making suggestions linked back to the Wikipedia snippet that a person could approve, edit, or reject. This is a flow for scaling up annotation of data that works pretty well, as it also sucks having a ton of the gaps in the structured data, if it's sitting fine there in the linked Wikipedia page.
- StableAlkyne 3y agoDid they ever create the bridge from Wikipedia to Wikidata? I remember hearing talk about it as a way of helping the lack of data. The problem I had with Wikidata a couple years ago was that it was usually an incomplete subset of Wikipedia's infoboxes. Checking again for m-xylene, https://m.wikidata.org/wiki/Q3234708 https://m.wikidata.org/wiki/Q3234708 You get physical property data and citations. Now compare that to the chem infobox in wikipedia: https://en.m.wikipedia.org/wiki/M-Xylene https://en.m.wikipedia.org/wiki/M-Xylene You get a lot more useful data, like the dipole moment and solubility (kinda important for a solvent like Xylene), and tons of other properties that Wikidata just doesn't have. All in the infobox. It's weird that they don't just copy the Wikipedia infobox for the chemicals in Wikidata. It's already there and organized. And frequently cited. Maybe it's more useful for other fields, but I can't think of a good use I'd get from the chemical section of Wikidata over the databases it cites or Wikipedia itself...
- raybb 3y agoIt would be really cool if there was a tool that could help extra data from say a news article and then populate wikidata with it after human review. I find the task of adding simple fields like date founded to be too many clicks with the default gui.
- huytersd 3y agoOnly if it is human validated and even then not really.
- loxias 3y agoThat word "augmented" is doing a lot of heavy lifting. LLMs don't "augment" data, they generate/hallucinate it. Sometimes they recall it.