3 ms·

AI can't solve novel problems yet

4 points by ddxv 1y ago

ddxv 1y ago
https://github.com/blacktop/ipsw/issues/668#issuecomment-2902870116 https://github.com/blacktop/ipsw/issues/668#issuecomment-290... I saw this hilarious exchange in one of the issues covering Apple’s recent changes that make it impossible to download IPA files. Recreating an iPhone’s authentication to Apple App Store is a bit of a cat and mouse game. Apple recently changed it’s backend, so now dozens of projects are trying to find ways to circumvent or crack the Apple App Store download authentication. Blacktop, out of likely boredom or curiosity, threw Claude at this issue for Claude to error out. To be fair, it’s likely impossible for Claude to have figured this issue out. The ‘fix’ will be something quite novel, for example perhaps reverse engineering a way to authenticate with modern Apple App Store using an old Windows version of iTunes that is still supported. Perhaps in the near future AI agents will be powerful enough to understand more complex environments, which will be interesting for these types of problems, but for now, it’s far from capable. For anyone interested in the issue, this one is active as well: https://github.com/majd/ipatool/issues/357 https://github.com/majd/ipatool/issues/357
NAHWheatCracker 1y ago
Are there any well-thought treatises on what "novel" means?
- ddxv 1y ago
  I was using it to mean new / not seen before problem. Perhaps that evokes a bit too grand concept? I know the issue I highlight is not that 'important' overall. Edit: You're right the 'problem' might not be new/novel, just that the solution is new.
  - NAHWheatCracker 1y ago
    I wasn't criticizing the choice of word, apologies. I'm interested if anyone has an idea of what it would mean for an AI to generate or do something "novel" in the sense of creatively new.
    - ddxv 1y ago
      Of course, I found the question interesting as well!
- beardyw 1y ago
  What "novel" means? I would say putting things together in a way which hasn't been done before. Just asked Gemini: There are 2 wolves and 2 sheep in a field. How many sheep will there be tomorrow? Answer: There will still be 2 sheep tomorrow. Needs some visualisation which LLMs don't really do well.

4 points by ddxv 1y ago

ddxv 1y ago

https://github.com/blacktop/ipsw/issues/668#issuecomment-2902870116 https://github.com/blacktop/ipsw/issues/668#issuecomment-290... I saw this hilarious exchange in one of the issues covering Apple’s recent changes that make it impossible to download IPA files. Recreating an iPhone’s authentication to Apple App Store is a bit of a cat and mouse game. Apple recently changed it’s backend, so now dozens of projects are trying to find ways to circumvent or crack the Apple App Store download authentication. Blacktop, out of likely boredom or curiosity, threw Claude at this issue for Claude to error out. To be fair, it’s likely impossible for Claude to have figured this issue out. The ‘fix’ will be something quite novel, for example perhaps reverse engineering a way to authenticate with modern Apple App Store using an old Windows version of iTunes that is still supported. Perhaps in the near future AI agents will be powerful enough to understand more complex environments, which will be interesting for these types of problems, but for now, it’s far from capable. For anyone interested in the issue, this one is active as well: https://github.com/majd/ipatool/issues/357 https://github.com/majd/ipatool/issues/357

NAHWheatCracker 1y ago

Are there any well-thought treatises on what "novel" means?

ddxv 1y ago
I was using it to mean new / not seen before problem. Perhaps that evokes a bit too grand concept? I know the issue I highlight is not that 'important' overall. Edit: You're right the 'problem' might not be new/novel, just that the solution is new.
- NAHWheatCracker 1y ago
  I wasn't criticizing the choice of word, apologies. I'm interested if anyone has an idea of what it would mean for an AI to generate or do something "novel" in the sense of creatively new.
  - ddxv 1y ago
    Of course, I found the question interesting as well!
beardyw 1y ago
What "novel" means? I would say putting things together in a way which hasn't been done before. Just asked Gemini: There are 2 wolves and 2 sheep in a field. How many sheep will there be tomorrow? Answer: There will still be 2 sheep tomorrow. Needs some visualisation which LLMs don't really do well.