3 ms·
> the more data you feed into an LLM, the smarter it should be in the response This is not obvious though.
by richrichie 2y ago
> the more data you feed into an LLM, the smarter it should be in the response
This is not obvious though.
- aurareturn 2y agoIt’s in theory. The more information you have, the better the decision in theory.
- threeseed 2y agoIt's quality not quantity. You need to have accurate, properly reasoned information for better decisions.
- coredog64 2y agoIt’s a good thing that SFDC and Slack are both well known for being a repository of high quality data. /sarc
- richrichie 2y agoNot quite. There are bounds on capacity of learning machines. https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_di...
- palad1n 2y agoAnyone got a TL;DR on this?
- vidarh 2y agoThink about an AI with a 1 bit model. If you feed that AI data that can't possibly be classified into less than 2 bits, it can't get it precisely right, no matter how much data you train it on, or what the 1 bit of the model represents. For any given size of system, there will be a ceiling on what it can learn to classify or predict with precision. I used "system" rather than "model" there for a reason: Memory in any form, such as context and RAG or API access to anything that can store and retrieve data affects the maximum - a turing machine can be implemented with a very small model + a loop if there's access to an external memory to act like the tape. But if the "tape" is limited, there will be some limitation on what the total system can precisely classify.