4 ms·

I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on severa

I wonder if the broad use of AI overviews on Google search results is having an impact. Maybe the numbers make it more profitable to use their compute on several billion searches a day rather than selling API access.

WarmWash 2mo ago
I think it's a safe bet that Google seems more interested in making a model that improves Google rather than making a model that improves workers. Fast, light weight, ok intelligence. Perfect for serving 20B+ prompts per day mostly surrounding banal human things. OAI and Anthropic's cloud spend can cover the revenue gap, as Google is already capturing a large chunk of those guy's revenue.
neutronicus 2mo ago
The AI mode on Google search is pretty impressive. Helped me figure out what a bunch of stuff I was seeing out the window was while traveling.
SadErn 2mo ago
Microsoft also seems to be working in this space. They recently released this: https://huggingface.co/microsoft/bitnet-embedding-0.6b https://huggingface.co/microsoft/bitnet-embedding-0.6b It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world. The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.
paxys 2mo ago
AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.
- amazingamazing 2mo ago
  based off what?
  - butlike 2mo ago
    vibes (coding)
bitshiftfaced 2mo ago
Not to mention internal use cases, such as prediction-related tasks like serving ads.
vikramkr 2mo ago
I doubt it's more profitable - ai overview is free/runs even when signed out in incognito, while the frontier models from openai and anthropic are nauseatingly expensive, especially for enterprise, and have a ton of users who are willing to cough up that money. Even if they aren't profitable because their cogs is even higher, they certainly have revenue with ai overview doesn't really have

WarmWash 2mo ago

I think it's a safe bet that Google seems more interested in making a model that improves Google rather than making a model that improves workers. Fast, light weight, ok intelligence. Perfect for serving 20B+ prompts per day mostly surrounding banal human things. OAI and Anthropic's cloud spend can cover the revenue gap, as Google is already capturing a large chunk of those guy's revenue.

neutronicus 2mo ago

The AI mode on Google search is pretty impressive. Helped me figure out what a bunch of stuff I was seeing out the window was while traveling.

SadErn 2mo ago

Microsoft also seems to be working in this space. They recently released this: https://huggingface.co/microsoft/bitnet-embedding-0.6b https://huggingface.co/microsoft/bitnet-embedding-0.6b It’s a small multilingual embedding model designed for things like search, RAG, and semantic similarity. It supports a fairly large context window and is designed to run efficiently on a CPU in a GPU starved world. The interesting part is that it builds on BitNet, using ternary weights of -1, 0, and 1 instead of the usual floating-point weights. That should make indexing and searching large amounts of text much cheaper without giving up too much accuracy.

paxys 2mo ago

AI overview is just a summarization of the top 2-3 results. Of course at Google scale that will still need a ton of compute, but the requirement for generating an overview is many orders of magnitude lower than asking the same question in Gemini.

amazingamazing 2mo ago
based off what?
- butlike 2mo ago
  vibes (coding)

bitshiftfaced 2mo ago

Not to mention internal use cases, such as prediction-related tasks like serving ads.

vikramkr 2mo ago

I doubt it's more profitable - ai overview is free/runs even when signed out in incognito, while the frontier models from openai and anthropic are nauseatingly expensive, especially for enterprise, and have a ton of users who are willing to cough up that money. Even if they aren't profitable because their cogs is even higher, they certainly have revenue with ai overview doesn't really have