3 ms·
The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible. Automating curation of the datasets with the lates
by valine 3y ago
The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible. Automating curation of the datasets with the latest and greatest GPT sounds very possible however. GPT-supervised learning could be the key to bootstrapping robust models, and I would be shocked if OpenAI isn't using GPT4 to filter their datasets as we speak.
- hutzlibu 3y ago"Automating curation of the datasets with the latest and greatest GPT sounds very possible however." It might also lead to recursive garbage generation. If the AI has flaws and those flaws are from the initial training, then I see no way, how the flawed data can ever generate clean data. "The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible. " And it is not impossible. It is just a lot of work. And maybe work we just have to do, if we want to create reliable AIs, that do not fail at random times. Now this alone might not completely remove hallucinations, but solid training data is just the base of it all. (and since we are living in the area of fake news, I welcome all efforts towards established facts and data out of general principle)
- AlanSE 3y agoFunny enough, in 2023, the option most likely to be viable for the impossible task of categorizing the training set would be to use a LLM. While I don't "trust" these AIs to give accurate information, it's probably within their capabilities to categorize by the above mentioned categories... then feed that back into another (very expensive) round of training, along with some theoretical developments to boot. I do think this is within the realm of possibility in the next ~1 year, but would be hard.
- hutzlibu 3y agoOh, I surely think LLM's could help with the task of curation. Maybe even spot lots of potential errors and flaws by themself, to get to the worst cases in the dataset faster. But to finally confirm or negate the actual data in question, there has to be at least one (not overworked) human in the loop (and many eyes would be better). Otherwise it will just reinforce the existing flaws.
- worrycue 3y agoI don’t think it’s just a matter of curating data sets for accuracy. The model seem to be able to invent falsehoods on its own based on no data whatsoever - e.g. I read that researchers asked an LLM about a made up biochemist and the LLM generated an entire fictional history for it.
- valine 3y agoMy hope would be that by labeling factual and fictional data it would coax the model into a state where it doesn’t invent facts outside of a creative writing context. Inventing falsehoods is practically the definition of creative writing, you don’t want the model to lose that ability. This is purely hypothetical, but I imagine that the internal mechanisms the model uses to invent stories out of thin air are different from the mechanisms the model uses to recall precise facts. Providing additional labels could help sharpen the division between creative and factual writing tasks.
- SrslyJosh 3y agoThe model can't "invent facts" because it doesn't know what facts are. It's a statistical model that encodes information about the text it was trained on. It has no higher level abstractions to inform its output. > I imagine that the internal mechanisms the model uses to invent stories out of thin air are different from the mechanisms the model uses to recall precise facts. Nope! If you input "What is the capital of the United States of America?", it's probably going to output the correct answer, because the question (and answer) probably appeared many times in the training data. If you input "What is the capital of the Gronk Republic?", whatever output you get is generated via the exact same mechanism.
- valine 3y agoThat’s the “LLMs are a fuzzy jpeg of the web” theory, and it’s far, far from the scientific consensus. No one knows for sure exactly what happens inside a 500B parameter model. I’ll just leave you with GPT4’s response to your question “What is the capital of the Gronk Republic”. “As an AI language model, I am not aware of any existing country or political entity called the "Gronk Republic." It could be a fictional or hypothetical place, in which case the capital would be determined by the creator or context in which it is mentioned. If you are referring to a real location, please provide more information or clarify the name of the place you are asking about.”
- dr_dshiv 3y ago> The datasets are massive, and curating them into buckets of fact/fiction by hand would be next to impossible No, only difficult.