29 ms·
To add to my comment above: Google DeepMind put out 16 videos about Gemini today, the total watch time at 1x speed is about 45 mins. I've now watched them all (
by tikkun 3y ago
To add to my comment above: Google DeepMind put out 16 videos about Gemini today, the total watch time at 1x speed is about 45 mins. I've now watched them all (at >1x speed).
In my opinion, the best ones are:
* https://www.youtube.com/watch?v=UIZAiXYceBI https://www.youtube.com/watch?v=UIZAiXYceBI - variety of video/sight capabilities
* https://www.youtube.com/watch?v=JPwU1FNhMOA https://www.youtube.com/watch?v=JPwU1FNhMOA - understanding direction of light and plants
* https://www.youtube.com/watch?v=D64QD7Swr3s https://www.youtube.com/watch?v=D64QD7Swr3s - multimodal understanding of audio
* https://www.youtube.com/watch?v=v5tRc_5-8G4 https://www.youtube.com/watch?v=v5tRc_5-8G4 - helping a user with complex requests and showing some of the 'thinking' it is doing about what context it does/doesn't have
* https://www.youtube.com/watch?v=sPiOP_CB54A https://www.youtube.com/watch?v=sPiOP_CB54A - assessing the relevance of scientific papers and then extracting data from the papers
My current context: API user of OpenAI, regular user of ChatGPT Plus (GPT-4-Turbo, Dall E 3, and GPT-4V), occasional user of Claude Pro (much less since GPT-4-Turbo with longer context length), paying user of Midjourney.
Gemini Pro is available starting today in Bard. It's not clear to me how many of the super impressive results are from Ultra vs Pro.
Overall conclusion: Gemini Ultra looks very impressive. But - the timing is disappointing: Gemini Ultra looks like it won't be widely available until ~Feb/March 2024, or possibly later.
> As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.
> Early next year, we’ll also launch Bard Advanced, a new, cutting-edge AI experience that gives you access to our best models and capabilities, starting with Gemini Ultra.
I hope that there will be a product available sooner than that without a crazy waitlist for both Bard Advanced, and Gemini Ultra API. Also fingers crossed that they have good data privacy for API usage, like OpenAI does (i.e. data isn't used to train their models when it's via API/playground requests).
- tikkun 3y agoMy general conclusion: Gemini Ultra > GPT-4 > Gemini Pro See Table 2 and Table 7 https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf https://storage.googleapis.com/deepmind-media/gemini/gemini_... (I think they're comparing against original GPT-4 rather than GPT-4-Turbo, but it's not entirely clear) What they've released today: Gemini Pro is in Bard today. Gemini Pro will be coming to API soon (Dec 13?). Gemini Ultra will be available via Bard and API "early next year" Therefore, as of Dec 6 2023: SOTA API = GPT-4, still. SOTA Chat assistant = ChatGPT Plus, still, for everything except video, where Bard has capabilities . ChatGPT plus is closely followed by Claude. (But, I tried asking Bard a question about a youtube video today, and it told me "I'm sorry, but I'm unable to access this YouTube content. This is possible for a number of reasons, but the most common are: the content isn't a valid YouTube link, potentially unsafe content, or the content does not have a captions file that I can read.") SOTA API after Gemini Ultra is out in ~Q1 2024 = Gemini Ultra, if OpenAI/Anthropic haven't released a new model by then SOTA Chat assistant after Bard Advanced is out in ~Q1 2024 = Bard Advanced, probably, assuming that OpenAI/Anthropic haven't released new models by then
- charcircuit 3y agoSOTA does not require being productionized. eg. GPT-3 was SOTA and it was not publicly accessible.
- nightski 3y agoThere has to be some way to verify the claim. Trust me bro isn't science.
- gpm 3y ago"Trust that I ran these tests with these results" is extremely common in science.
- nightski 3y agoIt's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data.
- verdverm 3y agoYou'll find the same thing in many academic/scientific papers
- deleted 3y ago[deleted]
- hughesjj 3y agoThe trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value
- guiomie 3y agoWatching these videos made me remember this cool demo Google did years ago where their earpods would auto translate in realtime a conversation between two people talking different languages. Turned out to be demo vaporware. Will this be the same thing?
- foota 3y agoAren't you talking about this? https://support.google.com/googlepixelbuds/answer/7573100?hl=en https://support.google.com/googlepixelbuds/answer/7573100?hl... (which exists?)
- ruleryak 3y agoI think they're getting at the idea that it was demoed as a real time babelfish, where a conversation simple happened between two people wearing the devices. Instead it was a glorified spoken dropdown selector for choosing the language, and a press and hold mechanism that just tied into the existing phone app without any actual changes or upgrades to that already available translation mechanism. The thought was that you'd simply start talking to each other and hear the other in your language as you go - not speak a block all at once, stop, translate, play back from your phone to them, stop, let them speak a whole reply at once while the phone listens to them, stop, translate, hear their response in your earpiece. Which basically meant the device itself didn't bring much if anything to the table that couldn't be done with any other headphones and doing the language select and start/stop recording on the phone itself.
- mirekrusin 3y agoYes it has a feel of coming from the same PR department as google glasses.
- DonHopkins 3y agoMaybe Robert Scoble will post pictures of himself using Gemini naked in the shower.
- spaceywilly 3y ago
- z3ncyberpunk 3y ago[flagged]
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- chatmasta 3y agoWait so it doesn't exist yet? Thanks for watching 45 minutes of video to figure that out for me. Why am I wasting my time reading this thread? Somebody please wake me up when I can talk to the thing by typing and dropping files into a chat box.
- rtsil 3y agoWhen I watch any of these videos, all the related videos on my right sidebar are from Google, 16 of which were uploaded at the same time as the one I'm watching. I've never seen the entire sidebar filled with the videos of a single channel before.
- GolfPopper 3y agoYeah. Dropping that blatant a weight on the algorithm is the most infuriating dark patterns I've noticed in a while.