3 ms·

Chess is a classic case which LLMs are structurally inadequate. They are structurally inadequate in a lot of cases. The real problem is it's really difficult

Chess is a classic case which LLMs are structurally inadequate. They are structurally inadequate in a lot of cases. The real problem is it's really difficult to know in advance if your case is one of them (HINT: Don't trust their cooking advice). They are simply not predictable and that makes them less than trustworthy. LLMs turn traditional computing on it's head --- instead of highly accurate results at low cost, you get questionable results at high cost.

PaulHoule 2y ago
They're structurally adequate for language translation in the sense that it is a fuzzy problem, people won't agree what a "good translation" is (translations of French writers like Baudrillard are bad [obscurantist] in my mind that they translate quotodienne to the $500 word quotidian instead of an everyday word like 'everyday') Even if an LLM based translator makes certain mistakes, other automatic translator may make even more mistakes. They aren't structurally adequate for the simpler problem of 'sort a list' because they can't make the N log N comparisons that it takes. It is hilarious to ask Copilot to sort a list in some order that isn't usual (like US states in alphabetical order) and see that it can't do it, and if you ask it if it can do it it will say it can, what probably it will get it right it says 'very high', then it goes ahead and tries and gets it wrong and if it is unlucky it even duplicates or drops items.
- jqpabc123 2y ago
  MS is on a mission to prove to large numbers of people that AI is not the next big revolution in computing. Copilot has all the potential Cortana had.
  - PaulHoule 2y ago
    For me Copilot is at the plateau of productivity. When I'm trying to figure out how to test for certain things with Mockito it gives me working examples faster than I'd figure out if it was me and my lonesome with the manual. If I don't know why something works it often gives a correct answer. It does take some critical thinking to use it successfully, but the same is true for StackOverflow. I've also gotten good advice on subjects such as voice acting. Trying to knock out quick prototypes with other code assistants I'd say I have a 50% success rate. For some circumscribed tasks I've had good luck with existing code bases but if the problem is something like "figure out how somebody managed to download a file they shouldn't have downloaded and make sure it never happens again" you'd be a fool to even try using one.
    - jqpabc123 2y ago
      For me Copilot is at the plateau of productivity. Here is a link to someone using similar words to describe Cortana. And we know how that turned out. https://www.fordway.com/ai-personal-assistants-from-hype-to-help https://www.fordway.com/ai-personal-assistants-from-hype-to-...

PaulHoule 2y ago

They're structurally adequate for language translation in the sense that it is a fuzzy problem, people won't agree what a "good translation" is (translations of French writers like Baudrillard are bad [obscurantist] in my mind that they translate quotodienne to the $500 word quotidian instead of an everyday word like 'everyday') Even if an LLM based translator makes certain mistakes, other automatic translator may make even more mistakes. They aren't structurally adequate for the simpler problem of 'sort a list' because they can't make the N log N comparisons that it takes. It is hilarious to ask Copilot to sort a list in some order that isn't usual (like US states in alphabetical order) and see that it can't do it, and if you ask it if it can do it it will say it can, what probably it will get it right it says 'very high', then it goes ahead and tries and gets it wrong and if it is unlucky it even duplicates or drops items.

jqpabc123 2y ago
MS is on a mission to prove to large numbers of people that AI is not the next big revolution in computing. Copilot has all the potential Cortana had.
- PaulHoule 2y ago
  For me Copilot is at the plateau of productivity. When I'm trying to figure out how to test for certain things with Mockito it gives me working examples faster than I'd figure out if it was me and my lonesome with the manual. If I don't know why something works it often gives a correct answer. It does take some critical thinking to use it successfully, but the same is true for StackOverflow. I've also gotten good advice on subjects such as voice acting. Trying to knock out quick prototypes with other code assistants I'd say I have a 50% success rate. For some circumscribed tasks I've had good luck with existing code bases but if the problem is something like "figure out how somebody managed to download a file they shouldn't have downloaded and make sure it never happens again" you'd be a fool to even try using one.
  - jqpabc123 2y ago
    For me Copilot is at the plateau of productivity. Here is a link to someone using similar words to describe Cortana. And we know how that turned out. https://www.fordway.com/ai-personal-assistants-from-hype-to-help https://www.fordway.com/ai-personal-assistants-from-hype-to-...