4 ms·

Apple says its latest AI model ReALM is even "better than OpenAI's GPT4

14 points by mgh2 3y ago

caseyy 3y ago
"Better" is such a subjective word in AI. I don't think it will be better than GPT4 in most areas. It's very difficult to imagine 3b or less being good in the general domain without some kind of current-day-math-subverting breakthroughs. However, something like Siri would need a very specialized model and a lot of value would come from its integration with apps. So in that sense, it could be a lot better at these specific tasks through training data curation than even very large models. 3B models can run quite well on consumer hardware, probably they'd work reasonably locally on iPhones. 250M would fly on an iPhone. I wonder if that's what they're aiming for. The models could also be so small because that makes sense in R&D, at this stage. But ultimately, most of the value to the user would probably come from how these models integrate with the app ecosystem. What's the point of making them generally smart if that will just cost exponentially more compute? From what we've seen with Llama2, for example, the compute costs go up exponentially for any sort of cleverness in the broad general domain of knowledge. In other words, the power law of the long tail/strongly diminishing returns apply for utility per kWh spent. Hope Apple is thinking this, I'd rather not tack on unnecessarily large compute costs to my iCloud subscription.
MichaelRo 3y ago
>> Apple says its latest AI model ReALM is even "better than OpenAI's GPT4 This offers no information whatsoever. It's all corporate bullshit PR. What can they even say? "Our AI is worse than OpenAI's?". In fact to paraphrase Seinfeld, I'd appreciate Apple's PR department (or any company's) to communicate this: "You should see our AI. It's the worst. Oh, yeah. It's the worst. It's the absolute worst there is. Whatever problem you've got, it'll be worse after you ask our AI. Yeah. It's just a... It's delirious. The AI's delirious."
mensetmanusman 3y ago
We have seen this before...
a2128 3y ago
The quality of this tweet and discussion is extremely disappointing. If you read the actual paper, you'll realize that Apple is not claiming to have discovered a magical breakthrough allowing their 80M model to somehow be competitive with 1T+ GPT4 on general tasks. This model does reference resolution. Given a query like "call the number" and a list of numbered on-screen elements, it'll tell you which elements the user may be referring to. It doesn't understand anything beyond just outputting number(s) referring to elements as such. You could consider "better than GPT4" to be true in this specific context, but this is the worst kind of truth. Then I've also trained many models that are "better than GPT4", only because GPT4 is not an AGI that is capable of everything in any modality. Windows 95 also shipped with an executable called "calc.exe" that's even better than GPT4

14 points by mgh2 3y ago

"Better" is such a subjective word in AI. I don't think it will be better than GPT4 in most areas. It's very difficult to imagine 3b or less being good in the general domain without some kind of current-day-math-subverting breakthroughs. However, something like Siri would need a very specialized model and a lot of value would come from its integration with apps. So in that sense, it could be a lot better at these specific tasks through training data curation than even very large models. 3B models can run quite well on consumer hardware, probably they'd work reasonably locally on iPhones. 250M would fly on an iPhone. I wonder if that's what they're aiming for. The models could also be so small because that makes sense in R&D, at this stage. But ultimately, most of the value to the user would probably come from how these models integrate with the app ecosystem. What's the point of making them generally smart if that will just cost exponentially more compute? From what we've seen with Llama2, for example, the compute costs go up exponentially for any sort of cleverness in the broad general domain of knowledge. In other words, the power law of the long tail/strongly diminishing returns apply for utility per kWh spent. Hope Apple is thinking this, I'd rather not tack on unnecessarily large compute costs to my iCloud subscription.

MichaelRo 3y ago

>> Apple says its latest AI model ReALM is even "better than OpenAI's GPT4 This offers no information whatsoever. It's all corporate bullshit PR. What can they even say? "Our AI is worse than OpenAI's?". In fact to paraphrase Seinfeld, I'd appreciate Apple's PR department (or any company's) to communicate this: "You should see our AI. It's the worst. Oh, yeah. It's the worst. It's the absolute worst there is. Whatever problem you've got, it'll be worse after you ask our AI. Yeah. It's just a... It's delirious. The AI's delirious."

mensetmanusman 3y ago

We have seen this before...

a2128 3y ago

The quality of this tweet and discussion is extremely disappointing. If you read the actual paper, you'll realize that Apple is not claiming to have discovered a magical breakthrough allowing their 80M model to somehow be competitive with 1T+ GPT4 on general tasks. This model does reference resolution. Given a query like "call the number" and a list of numbered on-screen elements, it'll tell you which elements the user may be referring to. It doesn't understand anything beyond just outputting number(s) referring to elements as such. You could consider "better than GPT4" to be true in this specific context, but this is the worst kind of truth. Then I've also trained many models that are "better than GPT4", only because GPT4 is not an AGI that is capable of everything in any modality. Windows 95 also shipped with an executable called "calc.exe" that's even better than GPT4