3 ms·
> No sign of what source material it was trained on though right? out of curiosity, does anyone do anything "useful" with that knowledge? it's not like people
by behnamoh 1y ago
> No sign of what source material it was trained on though right?
out of curiosity, does anyone do anything "useful" with that knowledge? it's not like people can just randomly train models..
- ToValueFunfetti 1y agoWould be useful for answering "is this novel or was it in the training data", but that's not typically what the point of open source is
- marci 1y agoWhen you're trully open source, you can make ethings like this: Today we introduce OLMoTrace, a one-of-a-kind feature in the Ai2 Playground that lets you trace the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace is a manifestation of Ai2’s commitment to an open ecosystem – open models, open data, and beyond. https://allenai.org/blog/olmotrace https://allenai.org/blog/olmotrace
- kreijstal 1y agoyou can do these same, except you would need to be a pirate website. It would even be better. except illegal. but it would be better.
- marci 1y agoThat is why the others can't provide stuff like this. RAG/Hallucination check. I just wish Allen.AI models had bigger context, 4k is too small nowadays.
- m00x 1y agoMany are speculating it was trained by o1/o3 for some of the initial reasoning.
- fulafel 1y agoAre there any widely used models that publish this? If not, then no I guess.
- DANmode 1y agoDepending on how you use "randomly", they absolutely can..?
- anonymoushn 1y agoIf labs provided the corpus and source code for training their tokenizers, it would be a lot easier to produce results about tokenizers. As it is, they provide neither, so it is impossible to compare different algorithms running on the same data if you also want to include the vocabs that are commonly used.