6 ms·
Gemini 1.5 outshines GPT-4-Turbo-128K on long code prompts, HVM author
- deleted 3y ago[deleted]
- anshumankmr 3y agoWhat I am not really liking about Gemini is how they haven't bundled this into an existing tier of Google One. I am paying for 2TB and I have to cough up extra to get access to the best model.
- behnamoh 3y agothey have, it's part of Google one.
- chaxor 3y agoYou may mean Gemini 1.0 Pro, which is the paid version of what used to be bard. Unfortunately Gemini Ultra is not available right now, and it seems Gemini 1.5 Pro (potentially similar to Ultra 1.0) may be available by waitlist.
- Deathmax 3y agoThe Google One AI plan does provide access to Gemini Advanced (backed by Gemini Ultra 1.0) at a significant premium to the current highest Google One plan: https://one.google.com/explore-plan/gemini-advanced https://one.google.com/explore-plan/gemini-advanced The free tier of Gemini (formerly Bard) has been on Gemini Pro 1.0 for a bit now.
- jug 3y agoGemini Pro is free at gemini.google.com, Gemini Ultra is the paid one. The remaining tier is Gemini Nano for on-device use. This has some serious implications for OpenAI as Gemini Pro 1.5 that is tested here indeed seems to beat their premium tier, assuming Google will keep using these tiers for what's paid and not.
- ukuina 3y agoNot with family sharing.
- AuthConnectFail 3y agoif you are primary payer, you still get it I think
- behnamoh 3y agoI'm excited about what Google comes up with but only as an intellectual curiosity. In practice, I'll never use it in my apps or workflow because I simply don't trust Google and they'll pull the plug any day with short notice. If Google's brand image wasn't tarnished due to their abandoning products that users loved, I'd probably have a different opinion.
- ukuina 3y agoAbstract the LLM calls into one layer, bundled with the prompt engineering for your use-case. Et viola, your application is now free from LLM lock-in.
- deeringc 3y agoWell, maybe at an API and prompt level. But if Google pull ahead in this space then you may become dependent on what it alone can do functionally. Even if you can trivially switch LLM and prompt, if the others aren't able to do something equivalent (or at the same level of quality) then you're still locked in. Until now we've basically had this situation with OpenAI.
- ukuina 3y agoWhy is vendor lock-in a concern if no other vendor offers that functionality?
- ryanjshaw 3y agoMy interest in LLMs lies in the static program analysis space and these results are genuinely exciting. LLM-guided analysis (e.g. invariant modelling or detecting bugs by identifying unusual implementation patterns) is going to be a thing. OpenAI is gradually turning into the Fisher-Price of AI. I cancelled my ChatGPT Pro subscription after 12mo due to increasingly low quality answers since ~Nov 2023. I occasionally try the API but the quality there is only slightly better than ChatGPT.
- deleted 3y ago[deleted]
- jimmywetnips 3y agoI keep trying alternatives to gpt 4 based on people shitting on it and you know what, every single time I try something else, I hit a bruh moment on the first try. I asked Gemini v1 2 days ago about programming and the first response was some nonsense about elections being a complex topic. if gpt 4 is Fisher Price then it's competitors are ill fitting kinder egg toys with missing pieces. https://i.imgur.com/gxh0mUZ.jpeg https://i.imgur.com/gxh0mUZ.jpeg
- Jensson 3y agoGemini in their interface has overly strong political correctness and safety which creates such strange responses, but if you avoid triggering that then it is pretty good. Edit: And in your screenshot you did get real responses, you see gemini generates several look at response 2, looks like what you wanted.
- scotty79 3y agoIt's so sad that we got AIs only after everybody got so sensitive about some utterances. I think the language itself was never supposed to be that serious as we are currently making it.
- darepublic 3y agolook forward to an LLM that can simulate talking to a mature highly knowledgable adult and not a painfully grinning corporate mannequin.
- tmikaeld 3y agoImpressive! It's too bad it's not possible to opt-out of model training on Googles Gemini (free/paid*) plans. Which includes: Conversations, Location, Feedback, Usage information And specifically this clause: "Please don't enter confidential information in your conversations or any data you wouldn't want a reviewer to see or Google to use to improve our products, services, and machine-learning technologies." Update: *There is a Gemini setting here https://myactivity.google.com/u/1/product/bard https://myactivity.google.com/u/1/product/bard but it does not explicitly state that it disables training, just activity history.
- throw03172019 3y agoYikes. No use case in medical then?
- tmikaeld 3y agoGoogle has a specific Medical AI called Med-PaLM that has more stringent PHI/HIPAA compliance.
- polishTar 3y agoFor enterprise, Google has something called Vertex AI which has a strict privacy policy but you have to pay just like any other typical cloud service. Presumably Gemini 1.5 will be available there soon.
- Deathmax 3y agoThis depends on which service you are using, any of the free products allow for training, but the paid services do not. Allows training on input: 1. Gemini (formerly Bard): https://gemini.google.com/ https://gemini.google.com/. Policy: https://support.google.com/gemini/answer/13594961 https://support.google.com/gemini/answer/13594961. Can opt out of training by disabling history. 2. AI Studio free plan: https://ai.google.dev/ https://ai.google.dev/. Does not allow training on input: 1. AI Studio paid plan: https://ai.google.dev/ https://ai.google.dev/. 2. Google Cloud Vertex AI: https://cloud.google.com/vertex-ai https://cloud.google.com/vertex-ai. Policy: https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_training https://cloud.google.com/vertex-ai/docs/generative-ai/data-g... 3. Gemini for Workspace (formerly Duet AI for Workspace).
- patrickhogan1 3y agoConfused. Many articles about how good Gemini 1.5 is. Yet I am in Generative AI Studio right now and I can't get access to it. If its so good, why make it so hard to access? 1. If you are an enterprise customer - the waitlist page is a blank black page as of 2/18/24 2. If you are non-enterprise customer - you can signup for waitlist.
- asah 3y agoI got in, no problem. Gemini Advanced (paid service) matches ChatGPT4 in my tests and wins in several, and has become my daily driver.
- patrickhogan1 3y agoGemini Advanced 1.0 model or 1.5? 1.5 is not available in the Gemini ChatGPT like interface. It is only available through AI Studio. But you have to signup for a waitlist.
- p1esk 3y agoHe’s talking about Gemini 1.5.
- IanCal 3y agoGemini advanced is gemini 1 ultra, but they're after gemini 1.5 pro.
- Jensson 3y agoBecause they started beta testing it a few days ago, these things doesn't happen in hours.
- timenova 3y agoOff topic, but has anyone used HVM? What do you use it for? It seems like a very interesting project with quite a steep learning curve, given interaction nets look like a whole new way of thinking.
- deleted 3y ago[deleted]
- checker659 3y agoI wish someone would feed it a disassembled binary and see if it can make sense of it.
- lz400 3y agoSeems like it wouldn't. In the OP post, Gemini was able to reason about that codebase through the _text_. For example, comments and function names. It doesn't really understand the code. So binary code would be gibberish to it.
- HeatrayEnjoyer 3y agoGPT 4 base could do some reverse engineering on binaries
- denzlwashington 3y agoSince OpenAI has started early to give access to developers so that they can create new software with their models, it says alot about the company, and they are truly commited to create better solutions. I think all other LLMs are way back in this space and will have a hard time as soon as OpenAI continues to the next model.
- croes 3y agoOpenAI can't simply release a next model. At some point they run out of training data and others can catch up.
- andrewstuart 3y agoGoogle AI just doesn't work as well as ChatGPT. Conversations don't feel right or natural. ChatGPT 4 doesn't work as well as ChatGPT 3.5. The little brother is smarter han the big brother. ChatGPT 3.5 works better than Eliza. No-one has heard of any other AIs. That's pretty much it - the entire state of AI early 2024.
- dieortin 3y agoChatGPT 4 is much better than 3.5, and Gemini 1.0 Ultra is slightly better than ChatGPT 4. This is for software related tasks.
- prmph 3y agoHow long until we have an AI compiler? Say I design a language, and instead to writing an implementation, I train an AI with tons of input/output examples, the kinda of errors to emit for violations of the grammar. Will be cool to try this out. I imagine it might be more feasible to transpilation rsther than real compilation.
- pptr 3y agoWhy design a language? Just compile pseudo-code.
- Craig_Rodriguez 3y ago[dead]
- whywhywhywhy 3y agoI mean does GPT4 turbo even “shine” at all? Always felt more like a GPT3.8 than a 4. Honestly suspicious if it even shares much with 4 as it disappoints more times than it impresses.
- bamboozled 3y agoSo the cat is out of the bag, other companies know how to build ChatGPT-4 equivalent models?
- TacticalCoder 3y ago> So the cat is out of the bag, other companies know how to build ChatGPT-4 equivalent models? Yup. If a french company, Mistral, can do it you know the cat is out of the bag (ok, ok, started by an ex- OpenAI person I think). I tried mistral-next and it is impressive. The "moat" at this point seems to be: "Invest and train models using $$$ of hardware and electricity". It's both exciting (nobody has an edge) and depressing (the edge consist in buying billions worth of NVidia AI chips).
- bamboozled 3y agoYes, but I think overall it’s a healthy situation to be in. The Nvidia part is whatever. The fact this tech is becoming g democratized is good.
- victords 3y ago> The fact this tech is becoming g democratized is good. The issue is that it's only democratized if you have the money.
- sangnoir 3y ago> If a french company, Mistral, can do it you know the cat is out of the bag What's with the France slander? France is not some technological backwater - it's given us BeOS[1], VLC and Fabrice Bellard. 1. I know Be Inc was an American company, but check the nationalities of its alumni.
- asadotzler 3y agoIf you have the compute time to pull it off. I'll wager these kinds of models are only really possible by half a dozen companies so "other companies" might more accurately be written "Google and a few other companies" or something like that.
- deleted 3y ago[deleted]