6 ms·
I didn’t test with all LLM out there, but all of thus I tested failed with something as basic as "What is the number of words in the sentence coming before the
by psychoslave 2y ago
I didn’t test with all LLM out there, but all of thus I tested failed with something as basic as "What is the number of words in the sentence coming before the next one? Please answer."
- itchyjunk 2y agoHow many humans have you tested this with?
- psychoslave 2y agoInteresting point. Would you please answer the question I was mentioning? :)
- asgeir 2y agoIn my experience, LLMs tend to perform better if you give them instructions before the data to be operated on. At least for the ~13b size models. So,something like: Please count the number of words in the following sentence. "What is the number of words in the sentence coming before the next one?" edit: Which might be an artifact of the training data always being in that kind of format.
- Thorham 2y ago14
- olalonde 2y agoGPT-4 (OpenAI): The sentence you're referring to is "What is the number of words in the sentence coming before the next one? Please answer." It contains 14 words.
- psychoslave 2y agoThanks. I don’t have access to this engine which for some reason is kept in a closed garden for richer people. ¯\_(ツ)_/¯
- wrsh07 2y agoYou can always use the API which is dirt cheap? Just put $5 on and access via the playground They have better data policies and your $5 will go way farther than a 1 month subscription
- mikeocool 2y agoInterestingly, chat gpt 4o gave me the answer 15.