9 ms·

Google Gemini has the worst LLM API

203 points by indigodaddy 2y ago

jauntywundrkind 2y ago
In general, it's just wild to see Google squander such an intense lead. In 2012, Google was far ahead of the world in making the vast majority of their offerings intensely API-first, intensely API accessible. It all changed in such a tectonic shift. The Google Plus/Google+ era was this weird new reality where everything Google did had to feed into this social network. But there was nearly no API available to anyone else (short of some very simple posting APIs), where Google flipped a bit, where the whole company stopped caring about the rest of the world and APIs and grew intensely focused on internal use, on themselves, looked only within. I don't know enough about the LLM situation to comment, but Google squandering such a huge lead, so clearly stopping caring about the world & intertwingularity, becoming so intensely internally focused was such a clear clear clear fall. There's the Google Graveyard of products, but the loss in my mind is more clearly that Google gave up on APIs long ago, and has never performed any clear acts of repentance for such a grevious mis-step against the open world, open possibilities, against closed & internal focus.
- simonw 2y ago
  With Gemini 2.5 (both Pro and Flash) Google have regained so much of that lost ground. Those are by far the best long-context models right now, extremely competitively priced and they have features like image mask segmentation that aren't available from other models yet: https://simonwillison.net/2025/Apr/18/gemini-image-segmentation/ https://simonwillison.net/2025/Apr/18/gemini-image-segmentat...
  - jasonfarnon 2y ago
    I think the commenter was saying google squandered its lead ("goodwill" is how I would refer to it) in providing open and interoperable services, not the more recent lead it squandered in AI. I agree with your point that they've made up a lot of that ground with gemini 2.5.
    - simonw 2y ago
      Yeah you're right, I should have read their comment more closely. Google's API's have a way steeper learning curve than is necessary. So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google. Their permission model is diabolically complex to figure out too - same vibes as AWS, Google even used the same IAM acronym.
      - Aeolun 2y ago
        I feel like the AWS model isn’t all that hard for most of their API’s. It’s just something you don’t really want to think about.
      - PantaloonFlames 2y ago
        > So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google. I don't see that dependency. With ANY of the APIs. They're all documented. I invoke them directly from within emacs . OR you can curl them. I almost never use the wrapper libraries. I agree with your point that the client libraries are large and complicated, for my tastes. But there's no inherent dependency of the API on the library. The dependency arrow points the other direction. The libraries are optional; and in my experience, you can find 3p libraries that are thinner and more targeted if you like.
      - paul-tharun 2y ago
        I sometimes feel the complexity is present by design to increase the switching cost. Once you understand it and set it up on a project, you are locked in, as the perceived cost of moving is too high.
      - xyzzy_plugh 1y ago
        This is bizarre to read. gRPC is used _widely_ outside Google. I'm not aware of any API that requires you to use gRPC. I've never found their permission model to be complicated at all, at least compared to AWS.
  - tyre 2y ago
    Gemini 2.5 Pro is so good. I’ve found that using it as the architect and orchestrator, then farming subtasks and computer use to sonnet, is the best ROI
    - PantaloonFlames 2y ago
      You can also farm out subtasks to the Gemini Flash models. For example using Aider, use Pro for the "strong" model and Flash for the weak model.
      - tyre 1y ago
        I’ve tried flash. Sonnet with computer use has been a blast. Limited anecdata led me to 2.5 Pro + 3.7 Sonnet, but this all moves so fast that it good to reevaluate regularly.
    - egamirorrim 2y ago
      OOI what's your preferred framework for that managing agent/child agents setup?
      - tyre 1y ago
        I use Roo Code. It’s very good.
  - candiddevmike 2y ago
    The models are great but the quotas are a real pain in the ass. You will be fighting other customers for capacity if you end up needing to scale. If you have serious Gemini usage in mind, you almost have to have a Google Cloud TAM to advocate for your usage and quotas.
    - chrisheecho 2y ago
      We have moved our quota system to Dynamic Shared Quota (https://cloud.google.com/vertex-ai/generative-ai/docs/quotas https://cloud.google.com/vertex-ai/generative-ai/docs/quotas) for 2.0+ models. There are no quotas in DSQ. If you need a guaranteed throughput there is an option to purchase Provisioned Throughput (https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput https://cloud.google.com/vertex-ai/generative-ai/docs/provis...).
      - egamirorrim 2y ago
        The thing is that the entry level of provisioned throughput is so high! I just want a reliable model experience for my small Dev team using models through Vertex but I don't think there's anything I can buy there to ensure it.
      - dist-epoch 2y ago
        While we are talking about quotas, can you maybe add an easy way of checking how much you've used/got left? Apparently now you need to use google-cloud-quotas to get the limit and google-cloud-monitoring to get the usage. VS Code copilot managed to implement the first part, getting the limit using gemini-2.5-pro, but when I asked gemini to implement the second part it said that integrating cloud-monitoring is too complex and it can't do it !!!!
- aaronbrethorst 2y ago
  Hubris. It seems similar, at least externally, to what happened at Microsoft in the late 90s/early 00s. I am convinced that a split-up of Microsoft would have been invigorating for the spin-offs, and the tech industry in general would have been better for it. Maybe we’ll get a do-over with Google.
- caturopath 2y ago
  I don't understand why Sundar Pichai hasn't been replaced. Google seems like it's been floundering with respect to its ability to innovate and execute in the past decade. To the extent that this Google has been a good maintenance org for their cash cows, even that might not be a good plan if they dropped the ball with AI.
  - huntertwo 2y ago
    Everybody’s thinking the same thing. He sucks.
  - rs186 2y ago
    Answer is simple: he keeps cash coming in and stock price rising. You can compare his performance to his predecessors and CEOs at other companies. That does not necessarily make him a "good" leader in your eyes, but good enough to the board.
  - HDThoreaun 1y ago
    googles worth 2 trillion dollars off the back of a website. I think investors are so out of their depth with tech that theyre cool with his mediocre performance
    - caturopath 1y ago
      Two websites and an ad business.
  - shawabawa3 1y ago
    Google is the leader in LLMs and self-driving cars, two of the biggest innovation areas in the last decade, so how exactly has it been floundering in its ability to innovate and execute?
    - caturopath 1y ago
      Google isn't "the leader" in LLMs. Despite a huge funnel to get users in, for intentional use they are a distant second place for consumers, fourth place for LLM APIs, and reputationally treated as an underdog to two tiny companies.
- harlysparks 1y ago
  Google's headcount (and internal red tap) grew significantly from 2012 to 2025. You're highlighting the fact that at some point in its massive growth, Google had to stop relentlessly pushing R&D and allocate leadership focus on addressing technical debt (or at least operational efficiency) that was a consequence of that growth.
simonw 2y ago
I still don't really understand what Vertex AI is. If you can ignore Vertex most of the complaints here are solved - the non-Vertex APIs have easy to use API keys, a great debugging tool (https://aistudio.google.com https://aistudio.google.com), a well documented HTTP API and good client libraries too. I actually use their HTTP API directly (with the ijson streaming JSON parser for Python) and the code is reasonably straight-forward: https://github.com/simonw/llm-gemini/blob/61a97766ff0873936ad80b30e83d12439736d126/llm_gemini.py#L384 https://github.com/simonw/llm-gemini/blob/61a97766ff0873936a... You have to be very careful when searching (using Google, haha) that you don't accidentally end up in the Vertext documentation though. Worth noting that Gemini does now have an OpenAI-compatible API endpoint which makes it very easy to switch apps that use an OpenAI client library over to backing against Gemini instead: https://ai.google.dev/gemini-api/docs/openai https://ai.google.dev/gemini-api/docs/openai Anthropic have the same feature now as well: https://docs.anthropic.com/en/api/openai-sdk https://docs.anthropic.com/en/api/openai-sdk
- mgraczyk 2y ago
  OpenAI compatible API is missing important parameters, for example I don't think there is a way to disable flash 2 thinking with it. Vertex AI is for grpc, service auth, and region control (amongst other things). Ensuring data remains in a specific region, allowing you to auth with the instance service account, and slightly better latency and ttft
  - minimaxir 2y ago
    From the linked docs: > If you want to disable thinking, you can set the reasoning effort to "none". For other APIs, you can set the thinking tokens to 0 and that also works.
    - mgraczyk 2y ago
      Wow thanks I did not know
      - logankilpatrick 1y ago
ryao 2y ago
I have not pushed my local commits to GitHub lately (and probably should), but my experience with the Gemini API so far has been relatively positive: https://github.com/ryao/gemini-chat https://github.com/ryao/gemini-chat The main thing I do not like is that token counting is rated limited. My local offline copies have stripped out the token counting since I found that the service becomes unusable if you get anywhere near the token limits, so there is no point in trimming the history to make it fit. Another thing I found is that I prefer to use the REST API directly rather than their Python wrapper. Also, that comment about 500 errors is obsolete. I will fix it when I do new pushes.
- yorick 2y ago
  It looks like you can use the gemma tokenizer to count tokens up to at least the 1.5 models. The docs claim that there's a local compute_tokens function in google-genai, but it looks like it just does an API call. Example for 1.5: https://github.com/googleapis/python-aiplatform/blob/main/vertexai/tokenization/_tokenizer_loading.py https://github.com/googleapis/python-aiplatform/blob/main/ve...
SmellTheGlove 2y ago
Google’s APIs are all kind of challenging to ramp up on. I’m not sure if it’s the API itself or the docs just feeling really fragmented. It’s hard to find what you’re looking for even if you use their own search engine.
- candiddevmike 2y ago
  The Google Cloud API library is meant to be pretty dead simple. While there are bugs, there's a good chance if something's not working it's because of overthinking or providing too many args. Alternatively, doing more advanced stuff and straying from the happy path may lead to dragons.
- PantaloonFlames 2y ago
  The problem I've had is not that the APIs are complicated but that there are so darn many of them. I agree the API docs are not high on the usability scale. No examples, just reference information with pointers to types, which embed other types, which use abstract descriptions. Figuring out what sort of json payload you need to send, can take...a bunch of effort.
- arccy 2y ago
  they're usually pretty well structured and actually follow design principles like https://cloud.google.com/apis/design https://cloud.google.com/apis/design and https://google.aip.dev/1 https://google.aip.dev/1 once it clicks, it's infinitely better than the AWS style GetAnythingGoes apis....
asadm 2y ago
I don't get the outrage. Just use their OpenAI endpoints: https://ai.google.dev/gemini-api/docs/openai https://ai.google.dev/gemini-api/docs/openai It's the best model out there.
- ramoz 2y ago
  I have no issues with their native structured outputs either. Other than confusing and partially incomplete documentation.
  - chrisheecho 2y ago
    Ramoz, good to hear that native Structured Outputs are working! But if the docs are 'confusing and partially incomplete,' that’s not a good DevEx. Good docs are non-negotiable. We are in the process of revamping the whole documentation site. Stay tuned, you will see something better than what we have today.
    - ramoz 1y ago
      Product idea for structured outputs: Dynamic Json field... like imagine if I want a custom schema generated (e.g. for new on-the-fly structured outputs).
      - chrisheecho 1y ago
        ooh i like!
- malshe 1y ago
  Thanks for sharing this. I did not know this existed
behnamoh 2y ago
Even their OAI-compatible API isn't fully compatible. Tools like Instructor have special-casing for Gemini...
lemming 2y ago
Additionally, there's no OpenAPI spec, so you have to generate one from their protobuf specs if you want to use that to generate a client model. Their protobuf specs live in a repo at https://github.com/googleapis/googleapis/tree/master/google/ai/generativelanguage https://github.com/googleapis/googleapis/tree/master/google/.... Now you might think that v1 would be the latest there, but you would be wrong - everyone uses v1beta (not v1, not v1alpha, not v1beta3) for reasons that are completely unclear. Additionally, this repo is frequently not up to date with the actual API (it took them ages to get the new thinking config added, for example, and their usage fields were out of date for the longest time). It's really frustrating.
- ezekiel68 2y ago
  Eh, you know. "Move fast and break things."
  - caturopath 2y ago
    I'm not sure "move fast" describes the situation.
    - ezekiel68 1y ago
      Hmm, the proliferation of branches, including some which seem perhaps more recent than "v1beta" made me imagine this could apply.
- chrisheecho 2y ago
  lemming, this is super helpful, thank you. We provide the genai SDK (https://github.com/googleapis/python-genai https://github.com/googleapis/python-genai) to reduce the learning curve in 4 languages (GA: Python, Go Preview: Node.JS, Java). The SDK works for all Gemini APIs provided by Google AI Studio (https://ai.google.dev/ https://ai.google.dev/) and Vertex AI.
  - egamirorrim 2y ago
    The way dependency resolution works in Java with the special, Google only, giant dynamic BOM resolver is hell on earth. We have to write code that round robins every region on retries to get past how overloaded/poorly managed vertex is (we're not hitting our quotas) and yes that's even with retry settings on the SDK. Read timeouts aren't configurable on the Vertex SDK.
rafram 2y ago
Site seems to be down - I can’t get the article to load - but by far the most maddening part of Vertex AI is the way it deals with multimodal inputs. You can’t just attach an image to your request. You have to use their file manager to upload the file, then make sure it gets deleted once you’re done. That would all still be OK-ish except that their JS library only accepts a local path, which it then attempts to read using the Node `fs` API. Serverless? Better figure out how to shim `fs`! It would be trivial to accept standard JS buffers. But it’s not clear that anyone at Google cares enough about this crappy API to fix it.
- Deathmax 2y ago
  > You can’t just attach an image to your request. You can? Google limits HTTP requests to 20MB, but both the Gemini API and Vertex AI API support embedded base64-encoded files and public URLs. The Gemini API supports attaching files that are uploaded to their Files API, and the Vertex AI API supports files uploaded to Google Cloud Storage.
  - rafram 2y ago
    Their JavaScript library didn’t support that as of whenever I tried.
    - simonw 2y ago
      I got their most recent JavaScript API library to work for images here: https://tools.simonwillison.net/gemini-mask https://tools.simonwillison.net/gemini-mask Here's the code: https://github.com/simonw/tools/blob/main/gemini-mask.html https://github.com/simonw/tools/blob/main/gemini-mask.html
- mofunnyman 2y ago
  Semi hugged.
- chrisheecho 2y ago
  That’s correct! You can send images through uploading either the Files API from Gemini API or Google Cloud Storage (GCS) bucket reference. What we DON’T have a sample on is sending images through bytes. Here is a screenshot of the code sample from the “Get Code” function in the Vertex AI studio. https://drive.google.com/file/d/1rQRyS4ztJmVgL2ZW35NXY0TW-S0E-p3x/view https://drive.google.com/file/d/1rQRyS4ztJmVgL2ZW35NXY0TW-S0... Let me create a feature request to get these samples in our docs because I could not find a sample too. Fixing it
bionhoward 2y ago
Also has the same customer noncompete copy pasted from ClosedAI. Not that anyone seemingly cares about the risk of lawsuits from Google for using Gemini in a way that happens to compete with random-Gemini-tentacle-123
deleted 2y ago
[deleted]
tom_m 2y ago
Doesn't matter much, Google already won the AI race. They had all the eyeballs already. There's a huge reason why they are getting slapped with anti-trust right now. The other companies aren't happy. I agree though, their marketing and product positioning is super confusing and weird. They are running their AI business in a very very very strange way. This has created a delay, I don't think opportunity for others, in their dominance in this space. Using Gemini inside BigQuery (this is via Vertex) is such a stupid good solution. Along with all of the other products that support BigQuery (datastream from cloudsql MySQL/postgres, dataform for query aggregation and transformation jobs, BigQuery functions, etc.), there's an absolutely insane amount of power to bring data over to Gemini and back out. It's literally impossible for OpenAI to compete because Google has all of the other ingredients here already and again, the user base. I'm surprised AWS didn't come out stronger here, weird.
- tom_m 2y ago
  Oh and it's not just Gemini, I'm sorry. It's Vertex. So it's other models as well. Those you train too.
- ashu1461 1y ago
  I don't think so, it might be true for their long tail customers, but most of the tech folks have not used Google Search / Gemini APIs in ages.
- harlysparks 1y ago
  They are getting slapped with anti-trust right now because they gave $10M to Kamala Harris and only $1M to Trump. That's it. Also, the AI race is a red queen race. There is no line on the sand that says "you are the ultimate winner", that's not how time works. And given that the vast majority of the internet is on AWS, NOT GCP, and that Gemini isn't even the most popular LLM among AI developers, I'm not sure you can even say that Google is the leader at this exact point in time.
  - tom_m 1y ago
    Yea, but Microsoft was slapped with anti-trust before too. These companies shrug it off. Yea, I guess it's not really much of a race. Because they are competing for something Google already largely has. In a few years, literally everyone is going to have the same AI and it'll become cheaper and cheaper to operate. I mean already, if you look at those leaderboards, many models are more or less the same. Training data (legal data) is running out. The algorithms are pushing their limits. It's not unlimited. It's not the second coming of Jesus as some people believe. It's a commodity and everyone is going to have it. So fast forward a little and think about that. The companies in the better position, the "winners," are going to be the same they always are. Those with the bigger user base and those with more data. That's Google. Yes, AWS too. Yes, Microsoft. The reason I'm bullish on Google is I suppose not because of Gemini itself. It's because of a super power like Google having their own LLM that performs as well as any other. I also believe OpenAI will go out of business or be acquired. They way way overspent, but they were also pioneers here a bit so they took on that burden. They over raised. That's painfully obvious after DeepSeek. They lost most of their top talent. They are being propped up because there's a LOT of money at stake. Not just for OpenAI but for other companies too. They are the face of AI so if they went belly up tomorrow then it'd start a wave of panic selling. People would think AI failed. So until some other successor steps in (Meta, Microsoft, Google, etc.), everyone including competitors will keep OpenAI on life support if need be. I don't think it helps that they foolishly bought Windsurf for $3B when you go and look at Roo Code hanging out there for free.
fumeux_fume 2y ago
I’m sorry have you used Azure? I’ve worked with all the major cloud providers and Google has its warts, but pales in comparison to the hoops Azure make you jump through to make a simple API call.
- ic_fly2 2y ago
  Azure API for LLM changes depending on what datacenter you are calling. It is bonkers. In fact it is so bad that at work we are hosting our own LLMs on azure GPU machines rather than use their API. (Which means we only have small models at much higher cost…)
simianwords 2y ago
Am I the only one who prefers a more serious approach to prefix caching? It is a powerful tool and having an endpoint dedicated to it and being able to control TTL's using parameters seems like the best approach. On the other hand the first two approaches from OpenAI and Anthropic are frankly bad. Automatically detecting what should be prefix cached? Yuck! And I can't even set my own TTL's in Anthropic API (feel free to correct me - a quick search revealed this). Serious features require serious approaches.
- simonw 2y ago
  > Automatically detecting what should be prefix cached? Yuck! Why don't you like that? I absolutely love it.
  - simianwords 1y ago
    I mean't that this is the only way to control prefix caching. I consider this a serious feature - if I were to make an application using prefix caching I would not consider OpenAI at all. I can't control what gets cached and for how long. Wouldn't you want to give more power to the developer? Prefix caching seems like an important enough concept to leak to the end user.
    - simonw 1y ago
      Gemini's approach to prefix caching requires me to pay per hour for keeping the cache populated. I have to do pretty sophisticated price modeling and load prediction to use that effectively. Anthropic require me to add explicit cache breakpoints to my prompts, which charge for writes to the cache. If I get that wrong it can be more expensive than if I left caching turned off entirely. With OpenAI I don't have to do any planning or optimistic guessing at all: if my app gets a spike in traffic the caching kicks in automatically and saves me money.
      - simianwords 1y ago
        that's fair - i have some app ideas for which i would like control over prefix caching. for example you may want to prompt cache entire chunks of enterprise data that don't change too often. the whole RAG application would be built over this concept - paying per hour for caching is sensible here. >With OpenAI I don't have to do any planning or optimistic guessing at all: if my app gets a spike in traffic the caching kicks in automatically and saves me money. i think these are completely different use cases. is this not different just from having a redis sitting in front of the LLM provider? fundamentally i feel like prompt caching is something i want to control and not have happen automatically; i want to use information i have over my (future) access patterns to save costs. for instance i might prompt cache a whole PDF and ask multiple questions. if i choose to prompt cache the PDF, i can save a non trivial amount of tokens processed. how can OpenAI's automatic approach help me here?
chrisheecho 2y ago
Hey there, I’m Chris Cho (x: chrischo_pm, Vertex PM focusing on DevEx) and Ivan Nardini (x: ivnardini, DevRel). We heard you and let us answer your questions directly as possible. First of all, thank you for your sentiment for our latest 2.5 Gemini model. We are so glad that you find the models useful! We really appreciate this thread and everyone for the feedback on Gemini/Vertex We read through all your comments. And YES, – clearly, we've got some friction in the DevEx. This stuff is super valuable, helps me to prioritize. Our goal is to listen, gather your insights, offer clarity, and point to potential solutions or workarounds. I’m going to respond to some of the comments given here directly on the thread
- moralestapia 2y ago
  This is so cringe. I hope it doesn't become a trend on this site.
  - tgv 1y ago
    It’s the US style, which has made its way across the pond too: you have to make upbeat noises to remove any suspicion you’re criticizing.
    - moralestapia 1y ago
      Unlike others ... you got it. It is incredibily lame for a gargantuan company like Google and their thousands of developers and PMs and this and that ... to come to a remote corner of the web to pretend they are doing what they should have done 10 years ago.
      - creatonez 1y ago
        Google should have cleaned up its Gemini API 10 years ago?
        moralestapia 1y ago
        >Chat, briefly, what does a PM at a company like Google do? "A Product Manager (PM) at Google is responsible for guiding the development of products from conception to launch. They identify user needs, define product vision and strategy, prioritize features, work with cross-functional teams (engineering, design, marketing), and ensure the product aligns with business goals. They act as the bridge between technical teams and stakeholders to deliver successful, user-focused solutions." Some might have ignored your question, but in the spirit of good conversation, I figured I’d share a quick explanation of what a PM does, just in case it helps!
franze 2y ago
yeah, also grounding with Google in Google 2.5 Pro does not ... deliver any URLs back, just the domains from where it grounded it response it should return vertexai urls that redirect to the sources, but doesn't do it in all cases (in non of mine) according to the docs plus you mandatory need to display an HTML fragment with search links that you are not allowed to edit basically a corporate infight as an API
Havoc 2y ago
Definitely designed by multiple teams with no coordination. The very generous free tier is pretty much the only reason I'm using it at all
mattw1810 2y ago
Their patchy JSON schema support for tool calls & structured generation is also very annoying… things like unions that you’d think are table stakes (and in fact work fine with both OpenAI and Anthropic) get rejected & you have to go reengineer your entire setup to accommodate it.
miki123211 2y ago
TBH, my biggest gripe with Google is that they seem to support a slightly different JSON schema format for structured outputs than everybody else. Where Open AI encourages (or even forces) you to use refs for embedding one object in another, Google wants you to embed directly, which is not only wasteful but incompatible with how libraries that abstract over model providers do it. My structured output code (which uses litellm under the hood, which converts from Pydantic models to JSON schemas), does not work with Google's models for that reason.
- intalentive 2y ago
  I used Gemini to write a function that recursively resolves all the refs. Not a big deal to convert your pydantic schemas.
msp26 2y ago
The linked blog is down. But agreed, I would especially like to see this particular thing fixed. > Property ordering > When you're working with JSON schemas in the Gemini API, the order of properties is important. By default, the API orders properties alphabetically and does not preserve the order in which the properties are defined (although the Google Gen Al SDKs may preserve this order). If you're providing examples to the model with a schema configured, and the property ordering of the examples is not consistent with the property ordering of the schema, the output could be rambling or unexpected.
kaycey2022 2y ago
The page 404s now. I wonder what was said. :(
- zodiakzz 2y ago
  https://archive.ph/20250504014835/https://venki.dev/notes/google-gemini-is-bad https://archive.ph/20250504014835/https://venki.dev/notes/go...
djohnston 2y ago
Google APIs have always been difficult for me to use. I find their documentation super fragmented, and they always build the same thing twice with awkward overlaps. In this case, it's Vertex and GenAI - two different SDKs with isomorphic data structures that are seemingly incompatible, but only sometimes. I don't understand how these things happen - but as usual I blame PMs trying to mark their territory and pissing all over everything. The new gemini models themselves though, are killer. The confusion is a small price to pay.
- harlysparks 1y ago
  The duplicative functions and overlapping features is a product of how their perf annual review cycle works and the way you have to play the game to get promotions as a SWE there. I think they've changed the setup in the last few years, but historically Google rewards building lots of beta versions of stuff with some commercial potential, throwing it over a wall and seeing what sticks.
- smel 1y ago
  This happen when you have 200k employees competing internally more than with outside competition.
- shresbm123 1y ago
  Haha as one of those PMs - thank you for appreciating the models. We do have a unified SDK and are working on making the docs more comprehensive and clearer. But do let keep giving us specific and candid feedback
stoicfungi 2y ago
Multiple SDKs, and the documentation and API responses are not consistent. Today I've spent hours just to make MCP & function calling work. It is really painful to work with.
bundie 1y ago
Why is the page 404-ing?
Squarex 1y ago
The link is not working anymore. Maybe the article has been deleted?
kmod 1y ago
The worst part to me is the privacy nightmare with AI Studio. It's essentially impossible to tell whether any particular API call will end up being included in their training data since this depends on properties that are stored elsewhere and are not available to the developer -- even a simple property such as "does this account have billing enabled" is oddly difficult to evaluate, and I was told by their support that because I at one point had any free credits on my account that it was a trial account and not a billed account even though I had a credit card attached and was being charged. I don't know if this is true and there is no way for me to find out. At some point they updated their privacy policy in regards to this, but instead of saying that this will cause them to train on your data, now the privacy policy says both that they will train on this data and that they will not train on this data, with no indication of which statement takes precedence over the other.
- shresbm123 1y ago
  Unless you are in the free tier of the API we do not train on your data. But let us make it clearer in the policy. If you would like to get more clarity on terms please DM me at @shresbm on X
sawyna 1y ago
My personal daily experience with this! I first used vertexai APIs because that's what they suggested, that Gemini APIs are not for production use. Then there comes the Google.generativeai. I don't remember the reason but they were pushing me to start using this library. Now it's all flashy google.genai libraries that they are pushing! I have figured that this is what I should use and this is the documentation that I should look for, because doing a Google search or using an LLM gives me so many confusing results. The only thing that works for sure is reading the library code. That's what I'm doing these days. For example, the documentation in one of those above libraries say that Gemini can read a document from cloud storage if you give it the uri. That doesn't work in google.genai library. I couldn't figure out why. I imagined maybe Gemini might need access to the cloud storage bucket, but I couldn't find any documentation as to how I can do that. I finally understood that I need to use the new file API and that uri works. Yes, I like Gemini model they are really good. But the library documentation can be significantly simpler.
humerahoneya123 1y ago
It’s wild how Gemini is leading the pack on model capabilities—context length, fine-tuning, multimodality—and yet using it feels like you’re solving a UX puzzle just to ship a basic feature. It’s almost ironic: Google builds some of the smartest models out there, but makes devs feel dumb trying to integrate them. Imagine the adoption surge if the experience was even half as smooth as OpenAI’s. Power without usability is just potential, not progress.
Max-Limelihood 1y ago
I'mma be honest, I think the API is great—EXCEPT THEY STILL DON'T HAVE LOGPROBS. ARGH.

203 points by indigodaddy 2y ago

In general, it's just wild to see Google squander such an intense lead. In 2012, Google was far ahead of the world in making the vast majority of their offerings intensely API-first, intensely API accessible. It all changed in such a tectonic shift. The Google Plus/Google+ era was this weird new reality where everything Google did had to feed into this social network. But there was nearly no API available to anyone else (short of some very simple posting APIs), where Google flipped a bit, where the whole company stopped caring about the rest of the world and APIs and grew intensely focused on internal use, on themselves, looked only within. I don't know enough about the LLM situation to comment, but Google squandering such a huge lead, so clearly stopping caring about the world & intertwingularity, becoming so intensely internally focused was such a clear clear clear fall. There's the Google Graveyard of products, but the loss in my mind is more clearly that Google gave up on APIs long ago, and has never performed any clear acts of repentance for such a grevious mis-step against the open world, open possibilities, against closed & internal focus.

simonw 2y ago
With Gemini 2.5 (both Pro and Flash) Google have regained so much of that lost ground. Those are by far the best long-context models right now, extremely competitively priced and they have features like image mask segmentation that aren't available from other models yet: https://simonwillison.net/2025/Apr/18/gemini-image-segmentation/ https://simonwillison.net/2025/Apr/18/gemini-image-segmentat...
- jasonfarnon 2y ago
  I think the commenter was saying google squandered its lead ("goodwill" is how I would refer to it) in providing open and interoperable services, not the more recent lead it squandered in AI. I agree with your point that they've made up a lot of that ground with gemini 2.5.
  - simonw 2y ago
    Yeah you're right, I should have read their comment more closely. Google's API's have a way steeper learning curve than is necessary. So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google. Their permission model is diabolically complex to figure out too - same vibes as AWS, Google even used the same IAM acronym.
    - Aeolun 2y ago
      I feel like the AWS model isn’t all that hard for most of their API’s. It’s just something you don’t really want to think about.
    - PantaloonFlames 2y ago
      > So many of their APIs depend on complex client libraries or technologies like GRPC that aren't used much outside of Google. I don't see that dependency. With ANY of the APIs. They're all documented. I invoke them directly from within emacs . OR you can curl them. I almost never use the wrapper libraries. I agree with your point that the client libraries are large and complicated, for my tastes. But there's no inherent dependency of the API on the library. The dependency arrow points the other direction. The libraries are optional; and in my experience, you can find 3p libraries that are thinner and more targeted if you like.
    - paul-tharun 2y ago
      I sometimes feel the complexity is present by design to increase the switching cost. Once you understand it and set it up on a project, you are locked in, as the perceived cost of moving is too high.
    - xyzzy_plugh 1y ago
      This is bizarre to read. gRPC is used _widely_ outside Google. I'm not aware of any API that requires you to use gRPC. I've never found their permission model to be complicated at all, at least compared to AWS.
- tyre 2y ago
  Gemini 2.5 Pro is so good. I’ve found that using it as the architect and orchestrator, then farming subtasks and computer use to sonnet, is the best ROI
  - PantaloonFlames 2y ago
    You can also farm out subtasks to the Gemini Flash models. For example using Aider, use Pro for the "strong" model and Flash for the weak model.
    - tyre 1y ago
      I’ve tried flash. Sonnet with computer use has been a blast. Limited anecdata led me to 2.5 Pro + 3.7 Sonnet, but this all moves so fast that it good to reevaluate regularly.
  - egamirorrim 2y ago
    OOI what's your preferred framework for that managing agent/child agents setup?
    - tyre 1y ago
      I use Roo Code. It’s very good.
- candiddevmike 2y ago
  The models are great but the quotas are a real pain in the ass. You will be fighting other customers for capacity if you end up needing to scale. If you have serious Gemini usage in mind, you almost have to have a Google Cloud TAM to advocate for your usage and quotas.
  - chrisheecho 2y ago
    We have moved our quota system to Dynamic Shared Quota (https://cloud.google.com/vertex-ai/generative-ai/docs/quotas https://cloud.google.com/vertex-ai/generative-ai/docs/quotas) for 2.0+ models. There are no quotas in DSQ. If you need a guaranteed throughput there is an option to purchase Provisioned Throughput (https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput https://cloud.google.com/vertex-ai/generative-ai/docs/provis...).
    - egamirorrim 2y ago
      The thing is that the entry level of provisioned throughput is so high! I just want a reliable model experience for my small Dev team using models through Vertex but I don't think there's anything I can buy there to ensure it.
    - dist-epoch 2y ago
      While we are talking about quotas, can you maybe add an easy way of checking how much you've used/got left? Apparently now you need to use google-cloud-quotas to get the limit and google-cloud-monitoring to get the usage. VS Code copilot managed to implement the first part, getting the limit using gemini-2.5-pro, but when I asked gemini to implement the second part it said that integrating cloud-monitoring is too complex and it can't do it !!!!
aaronbrethorst 2y ago
Hubris. It seems similar, at least externally, to what happened at Microsoft in the late 90s/early 00s. I am convinced that a split-up of Microsoft would have been invigorating for the spin-offs, and the tech industry in general would have been better for it. Maybe we’ll get a do-over with Google.
caturopath 2y ago
I don't understand why Sundar Pichai hasn't been replaced. Google seems like it's been floundering with respect to its ability to innovate and execute in the past decade. To the extent that this Google has been a good maintenance org for their cash cows, even that might not be a good plan if they dropped the ball with AI.
- huntertwo 2y ago
  Everybody’s thinking the same thing. He sucks.
- rs186 2y ago
  Answer is simple: he keeps cash coming in and stock price rising. You can compare his performance to his predecessors and CEOs at other companies. That does not necessarily make him a "good" leader in your eyes, but good enough to the board.
- HDThoreaun 1y ago
  googles worth 2 trillion dollars off the back of a website. I think investors are so out of their depth with tech that theyre cool with his mediocre performance
  - caturopath 1y ago
    Two websites and an ad business.
- shawabawa3 1y ago
  Google is the leader in LLMs and self-driving cars, two of the biggest innovation areas in the last decade, so how exactly has it been floundering in its ability to innovate and execute?
  - caturopath 1y ago
    Google isn't "the leader" in LLMs. Despite a huge funnel to get users in, for intentional use they are a distant second place for consumers, fourth place for LLM APIs, and reputationally treated as an underdog to two tiny companies.
harlysparks 1y ago
Google's headcount (and internal red tap) grew significantly from 2012 to 2025. You're highlighting the fact that at some point in its massive growth, Google had to stop relentlessly pushing R&D and allocate leadership focus on addressing technical debt (or at least operational efficiency) that was a consequence of that growth.

simonw 2y ago

I still don't really understand what Vertex AI is. If you can ignore Vertex most of the complaints here are solved - the non-Vertex APIs have easy to use API keys, a great debugging tool (https://aistudio.google.com https://aistudio.google.com), a well documented HTTP API and good client libraries too. I actually use their HTTP API directly (with the ijson streaming JSON parser for Python) and the code is reasonably straight-forward: https://github.com/simonw/llm-gemini/blob/61a97766ff0873936ad80b30e83d12439736d126/llm_gemini.py#L384 https://github.com/simonw/llm-gemini/blob/61a97766ff0873936a... You have to be very careful when searching (using Google, haha) that you don't accidentally end up in the Vertext documentation though. Worth noting that Gemini does now have an OpenAI-compatible API endpoint which makes it very easy to switch apps that use an OpenAI client library over to backing against Gemini instead: https://ai.google.dev/gemini-api/docs/openai https://ai.google.dev/gemini-api/docs/openai Anthropic have the same feature now as well: https://docs.anthropic.com/en/api/openai-sdk https://docs.anthropic.com/en/api/openai-sdk

mgraczyk 2y ago
OpenAI compatible API is missing important parameters, for example I don't think there is a way to disable flash 2 thinking with it. Vertex AI is for grpc, service auth, and region control (amongst other things). Ensuring data remains in a specific region, allowing you to auth with the instance service account, and slightly better latency and ttft
- minimaxir 2y ago
  From the linked docs: > If you want to disable thinking, you can set the reasoning effort to "none". For other APIs, you can set the thinking tokens to 0 and that also works.
  - mgraczyk 2y ago
    Wow thanks I did not know
    - logankilpatrick 1y ago

ryao 2y ago

I have not pushed my local commits to GitHub lately (and probably should), but my experience with the Gemini API so far has been relatively positive: https://github.com/ryao/gemini-chat https://github.com/ryao/gemini-chat The main thing I do not like is that token counting is rated limited. My local offline copies have stripped out the token counting since I found that the service becomes unusable if you get anywhere near the token limits, so there is no point in trimming the history to make it fit. Another thing I found is that I prefer to use the REST API directly rather than their Python wrapper. Also, that comment about 500 errors is obsolete. I will fix it when I do new pushes.

yorick 2y ago
It looks like you can use the gemma tokenizer to count tokens up to at least the 1.5 models. The docs claim that there's a local compute_tokens function in google-genai, but it looks like it just does an API call. Example for 1.5: https://github.com/googleapis/python-aiplatform/blob/main/vertexai/tokenization/_tokenizer_loading.py https://github.com/googleapis/python-aiplatform/blob/main/ve...

SmellTheGlove 2y ago

Google’s APIs are all kind of challenging to ramp up on. I’m not sure if it’s the API itself or the docs just feeling really fragmented. It’s hard to find what you’re looking for even if you use their own search engine.

candiddevmike 2y ago
The Google Cloud API library is meant to be pretty dead simple. While there are bugs, there's a good chance if something's not working it's because of overthinking or providing too many args. Alternatively, doing more advanced stuff and straying from the happy path may lead to dragons.
PantaloonFlames 2y ago
The problem I've had is not that the APIs are complicated but that there are so darn many of them. I agree the API docs are not high on the usability scale. No examples, just reference information with pointers to types, which embed other types, which use abstract descriptions. Figuring out what sort of json payload you need to send, can take...a bunch of effort.
arccy 2y ago
they're usually pretty well structured and actually follow design principles like https://cloud.google.com/apis/design https://cloud.google.com/apis/design and https://google.aip.dev/1 https://google.aip.dev/1 once it clicks, it's infinitely better than the AWS style GetAnythingGoes apis....

asadm 2y ago

I don't get the outrage. Just use their OpenAI endpoints: https://ai.google.dev/gemini-api/docs/openai https://ai.google.dev/gemini-api/docs/openai It's the best model out there.

ramoz 2y ago
I have no issues with their native structured outputs either. Other than confusing and partially incomplete documentation.
- chrisheecho 2y ago
  Ramoz, good to hear that native Structured Outputs are working! But if the docs are 'confusing and partially incomplete,' that’s not a good DevEx. Good docs are non-negotiable. We are in the process of revamping the whole documentation site. Stay tuned, you will see something better than what we have today.
  - ramoz 1y ago
    Product idea for structured outputs: Dynamic Json field... like imagine if I want a custom schema generated (e.g. for new on-the-fly structured outputs).
    - chrisheecho 1y ago
      ooh i like!
malshe 1y ago
Thanks for sharing this. I did not know this existed

behnamoh 2y ago

Even their OAI-compatible API isn't fully compatible. Tools like Instructor have special-casing for Gemini...

lemming 2y ago

Additionally, there's no OpenAPI spec, so you have to generate one from their protobuf specs if you want to use that to generate a client model. Their protobuf specs live in a repo at https://github.com/googleapis/googleapis/tree/master/google/ai/generativelanguage https://github.com/googleapis/googleapis/tree/master/google/.... Now you might think that v1 would be the latest there, but you would be wrong - everyone uses v1beta (not v1, not v1alpha, not v1beta3) for reasons that are completely unclear. Additionally, this repo is frequently not up to date with the actual API (it took them ages to get the new thinking config added, for example, and their usage fields were out of date for the longest time). It's really frustrating.

ezekiel68 2y ago
Eh, you know. "Move fast and break things."
- caturopath 2y ago
  I'm not sure "move fast" describes the situation.
  - ezekiel68 1y ago
    Hmm, the proliferation of branches, including some which seem perhaps more recent than "v1beta" made me imagine this could apply.
chrisheecho 2y ago
lemming, this is super helpful, thank you. We provide the genai SDK (https://github.com/googleapis/python-genai https://github.com/googleapis/python-genai) to reduce the learning curve in 4 languages (GA: Python, Go Preview: Node.JS, Java). The SDK works for all Gemini APIs provided by Google AI Studio (https://ai.google.dev/ https://ai.google.dev/) and Vertex AI.
- egamirorrim 2y ago
  The way dependency resolution works in Java with the special, Google only, giant dynamic BOM resolver is hell on earth. We have to write code that round robins every region on retries to get past how overloaded/poorly managed vertex is (we're not hitting our quotas) and yes that's even with retry settings on the SDK. Read timeouts aren't configurable on the Vertex SDK.

rafram 2y ago

Site seems to be down - I can’t get the article to load - but by far the most maddening part of Vertex AI is the way it deals with multimodal inputs. You can’t just attach an image to your request. You have to use their file manager to upload the file, then make sure it gets deleted once you’re done. That would all still be OK-ish except that their JS library only accepts a local path, which it then attempts to read using the Node `fs` API. Serverless? Better figure out how to shim `fs`! It would be trivial to accept standard JS buffers. But it’s not clear that anyone at Google cares enough about this crappy API to fix it.

Deathmax 2y ago
> You can’t just attach an image to your request. You can? Google limits HTTP requests to 20MB, but both the Gemini API and Vertex AI API support embedded base64-encoded files and public URLs. The Gemini API supports attaching files that are uploaded to their Files API, and the Vertex AI API supports files uploaded to Google Cloud Storage.
- rafram 2y ago
  Their JavaScript library didn’t support that as of whenever I tried.
  - simonw 2y ago
    I got their most recent JavaScript API library to work for images here: https://tools.simonwillison.net/gemini-mask https://tools.simonwillison.net/gemini-mask Here's the code: https://github.com/simonw/tools/blob/main/gemini-mask.html https://github.com/simonw/tools/blob/main/gemini-mask.html
mofunnyman 2y ago
Semi hugged.
chrisheecho 2y ago
That’s correct! You can send images through uploading either the Files API from Gemini API or Google Cloud Storage (GCS) bucket reference. What we DON’T have a sample on is sending images through bytes. Here is a screenshot of the code sample from the “Get Code” function in the Vertex AI studio. https://drive.google.com/file/d/1rQRyS4ztJmVgL2ZW35NXY0TW-S0E-p3x/view https://drive.google.com/file/d/1rQRyS4ztJmVgL2ZW35NXY0TW-S0... Let me create a feature request to get these samples in our docs because I could not find a sample too. Fixing it

bionhoward 2y ago

Also has the same customer noncompete copy pasted from ClosedAI. Not that anyone seemingly cares about the risk of lawsuits from Google for using Gemini in a way that happens to compete with random-Gemini-tentacle-123

deleted 2y ago

[deleted]

tom_m 2y ago

Doesn't matter much, Google already won the AI race. They had all the eyeballs already. There's a huge reason why they are getting slapped with anti-trust right now. The other companies aren't happy. I agree though, their marketing and product positioning is super confusing and weird. They are running their AI business in a very very very strange way. This has created a delay, I don't think opportunity for others, in their dominance in this space. Using Gemini inside BigQuery (this is via Vertex) is such a stupid good solution. Along with all of the other products that support BigQuery (datastream from cloudsql MySQL/postgres, dataform for query aggregation and transformation jobs, BigQuery functions, etc.), there's an absolutely insane amount of power to bring data over to Gemini and back out. It's literally impossible for OpenAI to compete because Google has all of the other ingredients here already and again, the user base. I'm surprised AWS didn't come out stronger here, weird.

tom_m 2y ago
Oh and it's not just Gemini, I'm sorry. It's Vertex. So it's other models as well. Those you train too.
ashu1461 1y ago
I don't think so, it might be true for their long tail customers, but most of the tech folks have not used Google Search / Gemini APIs in ages.
harlysparks 1y ago
They are getting slapped with anti-trust right now because they gave $10M to Kamala Harris and only $1M to Trump. That's it. Also, the AI race is a red queen race. There is no line on the sand that says "you are the ultimate winner", that's not how time works. And given that the vast majority of the internet is on AWS, NOT GCP, and that Gemini isn't even the most popular LLM among AI developers, I'm not sure you can even say that Google is the leader at this exact point in time.
- tom_m 1y ago
  Yea, but Microsoft was slapped with anti-trust before too. These companies shrug it off. Yea, I guess it's not really much of a race. Because they are competing for something Google already largely has. In a few years, literally everyone is going to have the same AI and it'll become cheaper and cheaper to operate. I mean already, if you look at those leaderboards, many models are more or less the same. Training data (legal data) is running out. The algorithms are pushing their limits. It's not unlimited. It's not the second coming of Jesus as some people believe. It's a commodity and everyone is going to have it. So fast forward a little and think about that. The companies in the better position, the "winners," are going to be the same they always are. Those with the bigger user base and those with more data. That's Google. Yes, AWS too. Yes, Microsoft. The reason I'm bullish on Google is I suppose not because of Gemini itself. It's because of a super power like Google having their own LLM that performs as well as any other. I also believe OpenAI will go out of business or be acquired. They way way overspent, but they were also pioneers here a bit so they took on that burden. They over raised. That's painfully obvious after DeepSeek. They lost most of their top talent. They are being propped up because there's a LOT of money at stake. Not just for OpenAI but for other companies too. They are the face of AI so if they went belly up tomorrow then it'd start a wave of panic selling. People would think AI failed. So until some other successor steps in (Meta, Microsoft, Google, etc.), everyone including competitors will keep OpenAI on life support if need be. I don't think it helps that they foolishly bought Windsurf for $3B when you go and look at Roo Code hanging out there for free.

fumeux_fume 2y ago

I’m sorry have you used Azure? I’ve worked with all the major cloud providers and Google has its warts, but pales in comparison to the hoops Azure make you jump through to make a simple API call.

ic_fly2 2y ago
Azure API for LLM changes depending on what datacenter you are calling. It is bonkers. In fact it is so bad that at work we are hosting our own LLMs on azure GPU machines rather than use their API. (Which means we only have small models at much higher cost…)

simianwords 2y ago

Am I the only one who prefers a more serious approach to prefix caching? It is a powerful tool and having an endpoint dedicated to it and being able to control TTL's using parameters seems like the best approach. On the other hand the first two approaches from OpenAI and Anthropic are frankly bad. Automatically detecting what should be prefix cached? Yuck! And I can't even set my own TTL's in Anthropic API (feel free to correct me - a quick search revealed this). Serious features require serious approaches.

simonw 2y ago
> Automatically detecting what should be prefix cached? Yuck! Why don't you like that? I absolutely love it.
- simianwords 1y ago
  I mean't that this is the only way to control prefix caching. I consider this a serious feature - if I were to make an application using prefix caching I would not consider OpenAI at all. I can't control what gets cached and for how long. Wouldn't you want to give more power to the developer? Prefix caching seems like an important enough concept to leak to the end user.
  - simonw 1y ago
    Gemini's approach to prefix caching requires me to pay per hour for keeping the cache populated. I have to do pretty sophisticated price modeling and load prediction to use that effectively. Anthropic require me to add explicit cache breakpoints to my prompts, which charge for writes to the cache. If I get that wrong it can be more expensive than if I left caching turned off entirely. With OpenAI I don't have to do any planning or optimistic guessing at all: if my app gets a spike in traffic the caching kicks in automatically and saves me money.
    - simianwords 1y ago
      that's fair - i have some app ideas for which i would like control over prefix caching. for example you may want to prompt cache entire chunks of enterprise data that don't change too often. the whole RAG application would be built over this concept - paying per hour for caching is sensible here. >With OpenAI I don't have to do any planning or optimistic guessing at all: if my app gets a spike in traffic the caching kicks in automatically and saves me money. i think these are completely different use cases. is this not different just from having a redis sitting in front of the LLM provider? fundamentally i feel like prompt caching is something i want to control and not have happen automatically; i want to use information i have over my (future) access patterns to save costs. for instance i might prompt cache a whole PDF and ask multiple questions. if i choose to prompt cache the PDF, i can save a non trivial amount of tokens processed. how can OpenAI's automatic approach help me here?

chrisheecho 2y ago

Hey there, I’m Chris Cho (x: chrischo_pm, Vertex PM focusing on DevEx) and Ivan Nardini (x: ivnardini, DevRel). We heard you and let us answer your questions directly as possible. First of all, thank you for your sentiment for our latest 2.5 Gemini model. We are so glad that you find the models useful! We really appreciate this thread and everyone for the feedback on Gemini/Vertex We read through all your comments. And YES, – clearly, we've got some friction in the DevEx. This stuff is super valuable, helps me to prioritize. Our goal is to listen, gather your insights, offer clarity, and point to potential solutions or workarounds. I’m going to respond to some of the comments given here directly on the thread

moralestapia 2y ago
This is so cringe. I hope it doesn't become a trend on this site.
- tgv 1y ago
  It’s the US style, which has made its way across the pond too: you have to make upbeat noises to remove any suspicion you’re criticizing.
  - moralestapia 1y ago
    Unlike others ... you got it. It is incredibily lame for a gargantuan company like Google and their thousands of developers and PMs and this and that ... to come to a remote corner of the web to pretend they are doing what they should have done 10 years ago.
    - creatonez 1y ago
      Google should have cleaned up its Gemini API 10 years ago?
      - moralestapia 1y ago
        >Chat, briefly, what does a PM at a company like Google do? "A Product Manager (PM) at Google is responsible for guiding the development of products from conception to launch. They identify user needs, define product vision and strategy, prioritize features, work with cross-functional teams (engineering, design, marketing), and ensure the product aligns with business goals. They act as the bridge between technical teams and stakeholders to deliver successful, user-focused solutions." Some might have ignored your question, but in the spirit of good conversation, I figured I’d share a quick explanation of what a PM does, just in case it helps!

franze 2y ago

yeah, also grounding with Google in Google 2.5 Pro does not ... deliver any URLs back, just the domains from where it grounded it response it should return vertexai urls that redirect to the sources, but doesn't do it in all cases (in non of mine) according to the docs plus you mandatory need to display an HTML fragment with search links that you are not allowed to edit basically a corporate infight as an API

Havoc 2y ago

Definitely designed by multiple teams with no coordination. The very generous free tier is pretty much the only reason I'm using it at all

mattw1810 2y ago

Their patchy JSON schema support for tool calls & structured generation is also very annoying… things like unions that you’d think are table stakes (and in fact work fine with both OpenAI and Anthropic) get rejected & you have to go reengineer your entire setup to accommodate it.

miki123211 2y ago

TBH, my biggest gripe with Google is that they seem to support a slightly different JSON schema format for structured outputs than everybody else. Where Open AI encourages (or even forces) you to use refs for embedding one object in another, Google wants you to embed directly, which is not only wasteful but incompatible with how libraries that abstract over model providers do it. My structured output code (which uses litellm under the hood, which converts from Pydantic models to JSON schemas), does not work with Google's models for that reason.

intalentive 2y ago
I used Gemini to write a function that recursively resolves all the refs. Not a big deal to convert your pydantic schemas.

msp26 2y ago

The linked blog is down. But agreed, I would especially like to see this particular thing fixed. > Property ordering > When you're working with JSON schemas in the Gemini API, the order of properties is important. By default, the API orders properties alphabetically and does not preserve the order in which the properties are defined (although the Google Gen Al SDKs may preserve this order). If you're providing examples to the model with a schema configured, and the property ordering of the examples is not consistent with the property ordering of the schema, the output could be rambling or unexpected.

kaycey2022 2y ago

The page 404s now. I wonder what was said. :(

zodiakzz 2y ago
https://archive.ph/20250504014835/https://venki.dev/notes/google-gemini-is-bad https://archive.ph/20250504014835/https://venki.dev/notes/go...

djohnston 2y ago

Google APIs have always been difficult for me to use. I find their documentation super fragmented, and they always build the same thing twice with awkward overlaps. In this case, it's Vertex and GenAI - two different SDKs with isomorphic data structures that are seemingly incompatible, but only sometimes. I don't understand how these things happen - but as usual I blame PMs trying to mark their territory and pissing all over everything. The new gemini models themselves though, are killer. The confusion is a small price to pay.

harlysparks 1y ago
The duplicative functions and overlapping features is a product of how their perf annual review cycle works and the way you have to play the game to get promotions as a SWE there. I think they've changed the setup in the last few years, but historically Google rewards building lots of beta versions of stuff with some commercial potential, throwing it over a wall and seeing what sticks.
smel 1y ago
This happen when you have 200k employees competing internally more than with outside competition.
shresbm123 1y ago
Haha as one of those PMs - thank you for appreciating the models. We do have a unified SDK and are working on making the docs more comprehensive and clearer. But do let keep giving us specific and candid feedback

stoicfungi 2y ago

Multiple SDKs, and the documentation and API responses are not consistent. Today I've spent hours just to make MCP & function calling work. It is really painful to work with.

bundie 1y ago

Why is the page 404-ing?

Squarex 1y ago

The link is not working anymore. Maybe the article has been deleted?

kmod 1y ago

The worst part to me is the privacy nightmare with AI Studio. It's essentially impossible to tell whether any particular API call will end up being included in their training data since this depends on properties that are stored elsewhere and are not available to the developer -- even a simple property such as "does this account have billing enabled" is oddly difficult to evaluate, and I was told by their support that because I at one point had any free credits on my account that it was a trial account and not a billed account even though I had a credit card attached and was being charged. I don't know if this is true and there is no way for me to find out. At some point they updated their privacy policy in regards to this, but instead of saying that this will cause them to train on your data, now the privacy policy says both that they will train on this data and that they will not train on this data, with no indication of which statement takes precedence over the other.

shresbm123 1y ago
Unless you are in the free tier of the API we do not train on your data. But let us make it clearer in the policy. If you would like to get more clarity on terms please DM me at @shresbm on X

sawyna 1y ago

My personal daily experience with this! I first used vertexai APIs because that's what they suggested, that Gemini APIs are not for production use. Then there comes the Google.generativeai. I don't remember the reason but they were pushing me to start using this library. Now it's all flashy google.genai libraries that they are pushing! I have figured that this is what I should use and this is the documentation that I should look for, because doing a Google search or using an LLM gives me so many confusing results. The only thing that works for sure is reading the library code. That's what I'm doing these days. For example, the documentation in one of those above libraries say that Gemini can read a document from cloud storage if you give it the uri. That doesn't work in google.genai library. I couldn't figure out why. I imagined maybe Gemini might need access to the cloud storage bucket, but I couldn't find any documentation as to how I can do that. I finally understood that I need to use the new file API and that uri works. Yes, I like Gemini model they are really good. But the library documentation can be significantly simpler.

humerahoneya123 1y ago

It’s wild how Gemini is leading the pack on model capabilities—context length, fine-tuning, multimodality—and yet using it feels like you’re solving a UX puzzle just to ship a basic feature. It’s almost ironic: Google builds some of the smartest models out there, but makes devs feel dumb trying to integrate them. Imagine the adoption surge if the experience was even half as smooth as OpenAI’s. Power without usability is just potential, not progress.

Max-Limelihood 1y ago

I'mma be honest, I think the API is great—EXCEPT THEY STILL DON'T HAVE LOGPROBS. ARGH.