5 ms·
Apple Researchers Reveal New AI System That Can Beat GPT-4
- potatoman22 3y agoIt beats GPT-4... at reference resolution. "ReALM reconstructs the visual layout of a screen using textual representations. This involves parsing on-screen entities and their locations to generate a textual format that captures the screen's content and structure. Apple researchers found that this strategy, combined with specific fine-tuning of language models for reference resolution tasks, significantly outperforms traditional methods, including the capabilities of OpenAI's GPT-4.*
- sharemywin 3y agoThis seems like the last thing a walled garden would want to exist. An agent that could interact with a screen for you.
- throwaway11460 3y agoWell Apple wants excellent products.
- manchmalscott 3y agoI would have agreed with you more readily a year or two ago. Here’s a quote from the current Sherman lawsuit against Apple: As Apple’s executives openly acknowledge: “In looking at it with hindsight, I think going forward we need to set a stake in the ground for what features we think are ‘good enough’ for the consumer. I would argue we’re already doing more than what would have been good enough. But we find it very hard to regress our product features YOY [year over year].” It sounds to me like they think their products are too excellent, and they want them to be worse actually.
- hackernoteng 3y agoApple needs to go back to the Steve Jobs era where they just released a final product with no rumors. Just "here it is" on stage, and available that day in the store. They need to replace Siri ASAP with a real AI LLM.
- debian3 3y agoSiri is such a mystery to me. How could it be so bad.
- newaccount74 3y agoSiri was released under Steve Jobs. It has always been bad.
- jaggs 3y agoFrom Anthropic Claude Opus: Imagine you have a picture book, and each page shows a different scene with various characters and objects. Now, think of a smart robot that can look at these pages and understand what's in them. Apple's scientists have created a robot called ReALM that does something similar but with computer screens. When ReALM looks at a screen, it doesn't just see an image. Instead, it reads the screen like a book, identifying all the different things on the screen and where they are located. ReALM then writes down what it sees in a special way, kind of like making a list of everything on the screen and giving each item a specific place. This helps ReALM understand the screen's content and how it's organized. By doing this and with some extra training, ReALM has become really good at a task called "reference resolution." This means that when you ask ReALM about something specific on the screen, it can quickly find and point out what you're asking for, even better than other smart robots like GPT-4. In short, ReALM is like a super-smart robot that can read computer screens, make a list of what it sees, and help you find things on the screen faster and better than ever before!
- alberth 3y agoJohnny 5 So basically “Johnny 5”. https://youtu.be/lv9owSY015w?feature=shared https://youtu.be/lv9owSY015w?feature=shared
- jaggs 3y agoBatteries not included, of course!
- reaperman 3y agoWe don't need an auto-generated summary LLM-spam on every post. I'm not saying you do this every post, but if every post got this it would be fairly obnoxious.
- Art9681 3y agoI find it to be useful. Would you feel any different if they read the article, thought about it, typed up a summary, edited it a few times, and posted it instead?