3 ms·
Not trying to be cynical here, but I am genuinely interested is there a reason why these LLM don't/can't/won't apply some deterministic algorithm? I mean, count
by hansmayer 1y ago
Not trying to be cynical here, but I am genuinely interested is there a reason why these LLM don't/can't/won't apply some deterministic algorithm? I mean, counting characters and such, we have solved those problems ages ago.
- dan-robertson 1y agoI think the intuition is that they don’t ‘know’ that they are bad at counting characters and such, so they answer the same way they answer most questions.
- hansmayer 1y agoWell, they can be made to use custom tools for writing to files and such, so I am not sure if that is the real reason? I have a feeling it is more because of trying to make this an "everything technology".
- kingkongjaffa 1y agoI suppose the codewriting tools could also just write code to do this job if prompted
- simonw 1y agoThey can. ChatGPT has been able to count characters/words etc flawlessly for a couple of years now if you tell it to "use your Python tool".
- hansmayer 1y agoFair enough. But why do I have to tell them that, should they not be able to figure it out themselves? If I show a 5-year kid once how to use colour pencils, I won't have to show them each time they want to make a drawing. This is the core weakness of the LLMs - you have to micromanage them so much, that it runs counter to the core promise that is being pushed since 3+ years now.
- Lerc 1y agoSpecifically for simple character level questions, if LLMs did that automatically, we would be inundated with stories about "AI model caught cheating" They are stuck in a place where the models are expected to do two things simultaneously. People want them to show the peak of pure AI ability while at the same time be the most useful they can be. Err too much on the side automatic use of tools and people will claim you're just faking it, fail to use tools sufficiently and people will claim that the AI is incapable of operations that any regular algorithm could do.
- hansmayer 1y agoAre you sure? Isn´t one aspect of intelligence being able to use, apply and develop tools? Isnt that the core feature that got humanity ahead of other mammals? As an early adopter, I couldn´t have cared less if AI was cheating in terms of strictly academic terms. I care about results. Lets say we´re working on something together and I ask you what is the 123921 multiplied by 1212. As the most natural thing you will dish out your calculator and give me the result. Do I care how you reached it? No, so as long as the result is correct, reliable, repeatable and quick - AND - I did not specifically ask you to perform the calculation specifically by hand or only with your mental faculties. So this is missing from those tools and because we have to remember to tell them for each and every use case HOW to do it, they are not intelligent.