6 ms·

Here's what happened: 1. Google rolled our AI summaries on all of their search queries, through some very tiny model 2. Given worldwide search volume, that mod

by jjani 1y ago

Here's what happened: 1. Google rolled our AI summaries on all of their search queries, through some very tiny model 2. Given worldwide search volume, that model now represents more than 50% of all queries if you throw it on a big heap with "intentional" LLM usage 3. Google gets to claim "the median is now 33x lower!", as the median is now that tiny model giving summaries nobody asked for It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread. Google is basing this all of "median" because there's orders of magnitudes difference betwen strong models (what most people think of when you talk AI) and tiny models, which Google uses "most" by virtue of running them for every single google search to produce the summaries. So the "median" will be whatever tiny model they use for those models. Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much. It's absurdly misleading and rather obvious, but it feels like most are very eager to latch on to this so they can tell themselves their usage and work (for the many here in AI or at Google) is all peachy. I've been reading this place for years and have never before seen such uncritical adoption of an obvious PR piece detached from reality.

deleted 1y ago
[deleted]
shwaj 1y ago
Are you sure? It wouldn’t shock me, but they specifically say “Gemini Apps”. I wasn’t familiar with the term, but a web search indicated that it has a specific meaning, and it doesn’t seem to me like web search AI summaries would be covered by it. Am I missing something?
tobr 1y ago
I’ve dramatically reduced my median calories per meal, by scheduling eight new meals a day, each consisting of one lettuce leaf.
jonas21 1y ago
What exactly are you basing this assertion on (other than your feelings)? Are you accusing Google of lying when they say in the technical report [1]: > This impact results from: A 33x reduction in per-prompt energy consumption driven by software efficiencies—including a 23x reduction from model improvements, and a 1.4x reduction from improved machine utilization. followed by a list of specific improvements they've made? [1] https://services.google.com/fh/files/misc/measuring_the_environmental_impact_of_delivering_ai_at_google_scale.pdf https://services.google.com/fh/files/misc/measuring_the_envi...
- esperent 1y ago
  Unless marketing blogs from any company specifically say what model they are talking about, we should always assume they're hiding/conflating/mislabeling/misleading in every way possible. This is corporate media literacy 101. The burden of proof is on Google here. If they've reduced gemini 2.5 energy use by 33x, they need to state that clearly. Otherwise a we should assume they're fudging the numbers, for example: A) they've chosen one particular tiny model for this number or B) it's a median across all models including the tiny one they use for all search queries EDIT: I've read over the report and it's B) as far as I can see Without more info, any other reading of this is a failing on the reader's part, or wishful thinking if they want to feel good about their AI usage. We should also be ready to change these assumptions if Google or another reputable party does confirm this applies to large models like Gemini 2.5, but should assume the least impressive possible reading until that missing info arrives. Even more useful info would be how much electricity Google uses per month, and whether that has gone down or continued to grow in the period following this announcement. Because total energy use across their whole AI product range, including training, is the only number that really matters.
  - mgraczyk 1y ago
    They did specifically say in the linked report
    - esperent 1y ago
      Here's the report. Could you tell me where in it you found a link to 33x reduction (or any large reduction) for any specific non-tiny model? Because all I can find is lots of references to "median Gemini". In fact, I would say they're being extremely careful in this paper not to mention any particular Google models with regards to energy reduction. https://services.google.com/fh/files/misc/measuring_the_environmental_impact_of_delivering_ai_at_google_scale.pdf https://services.google.com/fh/files/misc/measuring_the_envi...
      - mgraczyk 1y ago
RajT88 1y ago
Big tech seems all about the fluff. But, wasn't it always so? Wasn't it always so in business of all kinds? Why should we expect anything different? We should have been skeptical all along.
- camillomiller 1y ago
  I’ve been covering tech for 20 years. No, it wasn’t always like that. There was a sincere mutual respect between the companies and the media industry that I don’t see anymore. Both sides have their fault, but you know it’s not media that huperscaled and created gazillionaires by the score. Also, software is way more bendable to the emperors’ whims, and Google has become particularly hypocritical in the way it publicly represent itself.
  - rbinv 1y ago
    Agreed. Big tech is trying to become the media industry.
mgraczyk 1y ago
As others have pointed out, this is false. Google has made their models and hardware more efficient, you can read the linked report. Most of the efficiency comes from quantization, MoE, new attention techniques, and distillation (making smaller models useable in place of bigger models)
- jjani 1y ago
  - The report doesn't name any Gemini models at all, only competitors'. Wonder why that is? If the models got so much more efficient, they'd be eager to show this. - The report doesn't name any averages (means), only medians. Why oh why would they be doing this, when all other marketing pieces always use the average because outside of HN 99% of Joes on the street have no idea what a median is/how it differs from the mean? The average is much more relevant here when "measuring the environmental impact of AI inference". - The report doesn't define what any of the terms "Gemini Apps", "the Gemini AI assistant" or "Gemini Apps text prompt" concretely mean
  - scott_w 1y ago
    To be fair, the report explains their reasoning: they state the mean is too sensitive to outliers. Now, I do agree it would have been nice to demonstrate this, however it could be genuine.
    - jjani 1y ago
      That's a complete cop out. They didn't give the data to back this up.
      - scott_w 1y ago
        They definitely should have shown an example, or referenced something else that backs up their claim. I think it was you who made the good point that, when it comes to data usage, the mean may well give you more meaningful information because of the outliers! I can see the median being useful for answering what the cost of one more server/agent/whatever would be, but that’s not what this paper is asking.
user568439 1y ago
"It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread." It's very concerning that you claim this without previously fully reading and understanding Google's publication...
raincole 1y ago
It's not what the report says. > It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread. It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google. > Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much Yes, exactly, never mind that. The report is to compare against a data point from May 2024, before Gemini 2.5 Pro became a thing.
- deleted 1y ago
  [deleted]
- latexr 1y ago
  > make shit up on HN and be the top comment as long as it's to bash Google. I don’t think that’s fair. Same would’ve happened if it were Microsoft, or Apple, or Amazon. By now we’re all used to (and tired) of these tech giants lying to us and being generally shitty. Additionally, for decades we haven’t been able to trust reports from big companies which say “everything is fine, really” when they publish it themselves, about themselves, contradicting the general wisdom of something bad they’ve been doing. Put those together and you have the perfect combination; we’re primed to believe they’re trying to deceive us again, because that’s what happens most of the time. It has nothing to do with it being Google, they just happened to be the target this time.
- ksec 1y ago
  >It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google. Off topic. I wanted to say somewhat counterintuitively I often upvote / submit things I disagree with and dont downvote it as long as sub comments offer a good counter argument or explanation. Sometimes being top just meant that is what most people are thinking, and it being wrong and corrected is precisely why I upvote it and wish it stayed on top so others can learn.
deleted 1y ago
[deleted]
jsnell 1y ago
I know there's a lot of rebuttals to this statement already, but I think there's a simpler way of showing it is incorrect: Figure 2 in the paper shows the LMArena score of whatever model is used for "median" Gemini query. That score is consistent with Gemini Flash (probably 2.0, given the numbers are from May), not a "tiny model" used for summaries nobody is asking for.

by jjani 1y ago

deleted 1y ago

[deleted]

shwaj 1y ago

Are you sure? It wouldn’t shock me, but they specifically say “Gemini Apps”. I wasn’t familiar with the term, but a web search indicated that it has a specific meaning, and it doesn’t seem to me like web search AI summaries would be covered by it. Am I missing something?

tobr 1y ago

I’ve dramatically reduced my median calories per meal, by scheduling eight new meals a day, each consisting of one lettuce leaf.

jonas21 1y ago

What exactly are you basing this assertion on (other than your feelings)? Are you accusing Google of lying when they say in the technical report [1]: > This impact results from: A 33x reduction in per-prompt energy consumption driven by software efficiencies—including a 23x reduction from model improvements, and a 1.4x reduction from improved machine utilization. followed by a list of specific improvements they've made? [1] https://services.google.com/fh/files/misc/measuring_the_environmental_impact_of_delivering_ai_at_google_scale.pdf https://services.google.com/fh/files/misc/measuring_the_envi...

esperent 1y ago
Unless marketing blogs from any company specifically say what model they are talking about, we should always assume they're hiding/conflating/mislabeling/misleading in every way possible. This is corporate media literacy 101. The burden of proof is on Google here. If they've reduced gemini 2.5 energy use by 33x, they need to state that clearly. Otherwise a we should assume they're fudging the numbers, for example: A) they've chosen one particular tiny model for this number or B) it's a median across all models including the tiny one they use for all search queries EDIT: I've read over the report and it's B) as far as I can see Without more info, any other reading of this is a failing on the reader's part, or wishful thinking if they want to feel good about their AI usage. We should also be ready to change these assumptions if Google or another reputable party does confirm this applies to large models like Gemini 2.5, but should assume the least impressive possible reading until that missing info arrives. Even more useful info would be how much electricity Google uses per month, and whether that has gone down or continued to grow in the period following this announcement. Because total energy use across their whole AI product range, including training, is the only number that really matters.
- mgraczyk 1y ago
  They did specifically say in the linked report
  - esperent 1y ago
    Here's the report. Could you tell me where in it you found a link to 33x reduction (or any large reduction) for any specific non-tiny model? Because all I can find is lots of references to "median Gemini". In fact, I would say they're being extremely careful in this paper not to mention any particular Google models with regards to energy reduction. https://services.google.com/fh/files/misc/measuring_the_environmental_impact_of_delivering_ai_at_google_scale.pdf https://services.google.com/fh/files/misc/measuring_the_envi...
    - mgraczyk 1y ago

RajT88 1y ago

Big tech seems all about the fluff. But, wasn't it always so? Wasn't it always so in business of all kinds? Why should we expect anything different? We should have been skeptical all along.

camillomiller 1y ago
I’ve been covering tech for 20 years. No, it wasn’t always like that. There was a sincere mutual respect between the companies and the media industry that I don’t see anymore. Both sides have their fault, but you know it’s not media that huperscaled and created gazillionaires by the score. Also, software is way more bendable to the emperors’ whims, and Google has become particularly hypocritical in the way it publicly represent itself.
- rbinv 1y ago
  Agreed. Big tech is trying to become the media industry.

mgraczyk 1y ago

As others have pointed out, this is false. Google has made their models and hardware more efficient, you can read the linked report. Most of the efficiency comes from quantization, MoE, new attention techniques, and distillation (making smaller models useable in place of bigger models)

jjani 1y ago
- The report doesn't name any Gemini models at all, only competitors'. Wonder why that is? If the models got so much more efficient, they'd be eager to show this. - The report doesn't name any averages (means), only medians. Why oh why would they be doing this, when all other marketing pieces always use the average because outside of HN 99% of Joes on the street have no idea what a median is/how it differs from the mean? The average is much more relevant here when "measuring the environmental impact of AI inference". - The report doesn't define what any of the terms "Gemini Apps", "the Gemini AI assistant" or "Gemini Apps text prompt" concretely mean
- scott_w 1y ago
  To be fair, the report explains their reasoning: they state the mean is too sensitive to outliers. Now, I do agree it would have been nice to demonstrate this, however it could be genuine.
  - jjani 1y ago
    That's a complete cop out. They didn't give the data to back this up.
    - scott_w 1y ago
      They definitely should have shown an example, or referenced something else that backs up their claim. I think it was you who made the good point that, when it comes to data usage, the mean may well give you more meaningful information because of the outliers! I can see the median being useful for answering what the cost of one more server/agent/whatever would be, but that’s not what this paper is asking.

user568439 1y ago

"It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread." It's very concerning that you claim this without previously fully reading and understanding Google's publication...

raincole 1y ago

It's not what the report says. > It's very concerning that this marketing puff piece is being eaten up by HN of all places as evidenced by the other thread. It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google. > Never mind that Gemini 2.5 Pro, which is what everyone here would actually be using, may well consume >100x much Yes, exactly, never mind that. The report is to compare against a data point from May 2024, before Gemini 2.5 Pro became a thing.

deleted 1y ago
[deleted]
latexr 1y ago
> make shit up on HN and be the top comment as long as it's to bash Google. I don’t think that’s fair. Same would’ve happened if it were Microsoft, or Apple, or Amazon. By now we’re all used to (and tired) of these tech giants lying to us and being generally shitty. Additionally, for decades we haven’t been able to trust reports from big companies which say “everything is fine, really” when they publish it themselves, about themselves, contradicting the general wisdom of something bad they’ve been doing. Put those together and you have the perfect combination; we’re primed to believe they’re trying to deceive us again, because that’s what happens most of the time. It has nothing to do with it being Google, they just happened to be the target this time.
ksec 1y ago
>It's very concerning that you can just make shit up on HN and be the top comment as long as it's to bash Google. Off topic. I wanted to say somewhat counterintuitively I often upvote / submit things I disagree with and dont downvote it as long as sub comments offer a good counter argument or explanation. Sometimes being top just meant that is what most people are thinking, and it being wrong and corrected is precisely why I upvote it and wish it stayed on top so others can learn.

deleted 1y ago

[deleted]

jsnell 1y ago

I know there's a lot of rebuttals to this statement already, but I think there's a simpler way of showing it is incorrect: Figure 2 in the paper shows the LMArena score of whatever model is used for "median" Gemini query. That score is consistent with Gemini Flash (probably 2.0, given the numbers are from May), not a "tiny model" used for summaries nobody is asking for.