2 ms·
If there's one thing that's becoming clear in the open source LLM world, it's that the dataset really is the 'secret sauce' for LLMs. There are endless combinat
by taneq 3y ago
If there's one thing that's becoming clear in the open source LLM world, it's that the dataset really is the 'secret sauce' for LLMs. There are endless combinations of various datasets plus foundation model plus training approach, and by far the key determinant of end model performance seems to be the dataset used.
- satvikpendem 3y ago> it's that the dataset really is the 'secret sauce' alwayshasbeen.jpg There have been articles about how "data is the new oil" for a couple of decades now, with the first reference I could find being from British mathematician Clive Humby in 2006 [0]. The fact that it rings even more true in the age of LLMs is simply just another transformation of the fundamental data underneath. [0] https://en.wikipedia.org/wiki/Clive_Humby#cite_ref-10 https://en.wikipedia.org/wiki/Clive_Humby#cite_ref-10
- still_grokking 3y ago2006? https://en.wikipedia.org/wiki/Scientia_potentia_est https://en.wikipedia.org/wiki/Scientia_potentia_est
- sharkoz 3y agoThe best answer to this is https://www.youtube.com/watch?v=ab6GyR_5N6c https://www.youtube.com/watch?v=ab6GyR_5N6c :)
- satvikpendem 3y ago> There have been articles about how "data is the new oil" for a couple of decades now, with the first reference I could find being from British mathematician Clive Humby in 2006 I am specifically referring to the phrase I quoted, not some more abstract sentiment.