3 ms·

Is search really that costly to run? $10/1000 searches seems really pricey. I'm wondering if these costs will come down in a few years.

by jarbus 1y ago

tuyguntn 1y ago
they will come down, because up until recently consumers were not paying directly for searches, with the LLM which has a cutoff date in the past and hallucinations, search got popular paid API. Popularity will grow even more, hence competition will increase and prices will change eventually
- AznHisoka 1y ago
  I dont think that will be true. What competition? Google, Bing, and.. Kagi? (And only one of those have a far superior index/algo than the others)
jsnell 1y ago
Yes. The Bing Search API is priced at $15/1k queries in the cheapest tier, Brave API is $9 at the non-toy tier, Google's pricing for a general search API is unknown but their Search grounding in Gemini costs $35/1k queries. Search API prices have been going up, not down, over time. The opposite of LLMs, which have gotten 1000x cheaper over the last two years.
- jwr 1y ago
  > Google's pricing for a general search API As I discovered recently, and much to my surprise, Google does not offer a "general search API", at least not officially. There is a "custom search" API that sounds like web search, but isn't: it offers a subset of the index, which is not immediately apparent. Confusing and misleading labeling there. Bing offers something a bit better, but I recently ended up trying the Kagi API, and it is the best thing I found so far. Expensive ($25/1000), but works well.
  - formercoder 1y ago
    I work at Google but not on this. We do offer Gemini with Google Search grounding which is similar to a search API.
    - QuadmasterXLII 1y ago
      ??????
    - teeklp 1y ago
      How much do you pay people to use this?
  - jsnell 1y ago
    There are multiple search engines known to be based on Google's API (Startpage, Leta, Kagi), so that product definitely exists. But it exciting that's all we know. They indeed do not publish anything about it. We don't know the price, the terms, or even the name.
  - camkego 1y ago
    Do you have any references to the point that the Google Custom Search API is for a subset of the regular Google search index?
    - ricw 1y ago
      No reference here but found this out the hard way too. Google search Ali is Utterly useless in fact and entirely different search results vs using the web. Bing is better. Haven’t tried ksgi yet
    - jwr 1y ago
      "References"? :-) This is a corporation we're talking about, and Google at that. Layers upon layers of obscurity, "strategic decisions" and discontinued products. Try it and you'll see — there is no official Search API and the Custom Search API is quite poor and not usable in most scenarios.
- ColinHayhurst 1y ago
  Excuse the self-promotion but Mojeek is £3/1,000: https://www.mojeek.com/services/search/web-search-api/ https://www.mojeek.com/services/search/web-search-api/
  - firtoz 1y ago
    > Can I store data obtained through the API? > You can store results on Business plan and optionally on the Enterprise plan. For other plans, you may store the results for 1 hour to enable caching. Curious... I can understand that this may be a defensive action, however, feels unenforceable. And in some cases impractical for the user, after seeing this I may keep looking for alternatives for example because it's not clear to me if I have a chat history that has the search results in one of the messages, do I have to have a kind of mechanism to clean those out or something?
AznHisoka 1y ago
If you want an unofficial API, most data providers usually charge $4/1000 queries or so. By unofficial, I mean they just scrape whats in Google and return that to you. So thats the benchmark I use, which means the cost here is around 2x that. As far as I know, the pricing really hasnt gone down over the years. If anything it has gone up because Google is increasingly making it harder for these providers
- Manouchehri 1y ago
  That seems expensive. For 100 results per query, serper.dev is $2/1000 queries and Bright Data is $1.5/1000 queries.
  - AznHisoka 1y ago
    Sorry, got this off by a multiple. Yes, pricing is around that. So these “official” APIs are much more expensive.
  - jbellis 1y ago
    I'm not sure that's correct -- the first party APIs are priced per query but BD is per 1k results. Not immediately obvious what they count as a "result" tho.
    - Manouchehri 1y ago
      It's really poor wording. Bright Data does indeed consider 100 results in a single request to be a single billed "result" event, billed at $1.5/1000 requests. I always set 100 results per request from Bright Data, and I can see my bill indeed says `SERP Requests: x reqs @ 1.5 $/CPM` (where `x` is the number of requests I've made, not x * 100). https://docs.brightdata.com/scraping-automation/serp-api/faqs#i-saw-that-google-is-making-some-changes-to-serp-results-are-you-going-to-continue-to-support-the-serp-num-parameter:so-will-there-be-any-change-to-pricing-due-to-this-new-feature https://docs.brightdata.com/scraping-automation/serp-api/faq... For serper.dev, they consider 10 results to be 1 "credit", and 20 to 100 results to be 2 "credits". They bill at $50/50,000 credits, so it becomes $1/1000 requests if you are okay with just 10 results per request, or $2/1000 requests if you want 100 results per request. (Both providers here scale pricing with larger volumes, just trying to compare the easiest price point for those getting started.)
OxfordOutlander 1y ago
Openai search mode is $30-50 per 1000 depending on low-high context Gemini is $30/1000 So Anthropic is actually the cheapest. For context, exa is $5 / 1000.
deleted 1y ago
[deleted]

Is search really that costly to run? $10/1000 searches seems really pricey. I'm wondering if these costs will come down in a few years.

by jarbus 1y ago

Is search really that costly to run? $10/1000 searches seems really pricey. I'm wondering if these costs will come down in a few years.

tuyguntn 1y ago

they will come down, because up until recently consumers were not paying directly for searches, with the LLM which has a cutoff date in the past and hallucinations, search got popular paid API. Popularity will grow even more, hence competition will increase and prices will change eventually

AznHisoka 1y ago
I dont think that will be true. What competition? Google, Bing, and.. Kagi? (And only one of those have a far superior index/algo than the others)

jsnell 1y ago

Yes. The Bing Search API is priced at $15/1k queries in the cheapest tier, Brave API is $9 at the non-toy tier, Google's pricing for a general search API is unknown but their Search grounding in Gemini costs $35/1k queries. Search API prices have been going up, not down, over time. The opposite of LLMs, which have gotten 1000x cheaper over the last two years.

jwr 1y ago
> Google's pricing for a general search API As I discovered recently, and much to my surprise, Google does not offer a "general search API", at least not officially. There is a "custom search" API that sounds like web search, but isn't: it offers a subset of the index, which is not immediately apparent. Confusing and misleading labeling there. Bing offers something a bit better, but I recently ended up trying the Kagi API, and it is the best thing I found so far. Expensive ($25/1000), but works well.
- formercoder 1y ago
  I work at Google but not on this. We do offer Gemini with Google Search grounding which is similar to a search API.
  - QuadmasterXLII 1y ago
    ??????
  - teeklp 1y ago
    How much do you pay people to use this?
- jsnell 1y ago
  There are multiple search engines known to be based on Google's API (Startpage, Leta, Kagi), so that product definitely exists. But it exciting that's all we know. They indeed do not publish anything about it. We don't know the price, the terms, or even the name.
- camkego 1y ago
  Do you have any references to the point that the Google Custom Search API is for a subset of the regular Google search index?
  - ricw 1y ago
    No reference here but found this out the hard way too. Google search Ali is Utterly useless in fact and entirely different search results vs using the web. Bing is better. Haven’t tried ksgi yet
  - jwr 1y ago
    "References"? :-) This is a corporation we're talking about, and Google at that. Layers upon layers of obscurity, "strategic decisions" and discontinued products. Try it and you'll see — there is no official Search API and the Custom Search API is quite poor and not usable in most scenarios.
ColinHayhurst 1y ago
Excuse the self-promotion but Mojeek is £3/1,000: https://www.mojeek.com/services/search/web-search-api/ https://www.mojeek.com/services/search/web-search-api/
- firtoz 1y ago
  > Can I store data obtained through the API? > You can store results on Business plan and optionally on the Enterprise plan. For other plans, you may store the results for 1 hour to enable caching. Curious... I can understand that this may be a defensive action, however, feels unenforceable. And in some cases impractical for the user, after seeing this I may keep looking for alternatives for example because it's not clear to me if I have a chat history that has the search results in one of the messages, do I have to have a kind of mechanism to clean those out or something?

AznHisoka 1y ago

If you want an unofficial API, most data providers usually charge $4/1000 queries or so. By unofficial, I mean they just scrape whats in Google and return that to you. So thats the benchmark I use, which means the cost here is around 2x that. As far as I know, the pricing really hasnt gone down over the years. If anything it has gone up because Google is increasingly making it harder for these providers

Manouchehri 1y ago
That seems expensive. For 100 results per query, serper.dev is $2/1000 queries and Bright Data is $1.5/1000 queries.
- AznHisoka 1y ago
  Sorry, got this off by a multiple. Yes, pricing is around that. So these “official” APIs are much more expensive.
- jbellis 1y ago
  I'm not sure that's correct -- the first party APIs are priced per query but BD is per 1k results. Not immediately obvious what they count as a "result" tho.
  - Manouchehri 1y ago
    It's really poor wording. Bright Data does indeed consider 100 results in a single request to be a single billed "result" event, billed at $1.5/1000 requests. I always set 100 results per request from Bright Data, and I can see my bill indeed says `SERP Requests: x reqs @ 1.5 $/CPM` (where `x` is the number of requests I've made, not x * 100). https://docs.brightdata.com/scraping-automation/serp-api/faqs#i-saw-that-google-is-making-some-changes-to-serp-results-are-you-going-to-continue-to-support-the-serp-num-parameter:so-will-there-be-any-change-to-pricing-due-to-this-new-feature https://docs.brightdata.com/scraping-automation/serp-api/faq... For serper.dev, they consider 10 results to be 1 "credit", and 20 to 100 results to be 2 "credits". They bill at $50/50,000 credits, so it becomes $1/1000 requests if you are okay with just 10 results per request, or $2/1000 requests if you want 100 results per request. (Both providers here scale pricing with larger volumes, just trying to compare the easiest price point for those getting started.)

OxfordOutlander 1y ago

Openai search mode is $30-50 per 1000 depending on low-high context Gemini is $30/1000 So Anthropic is actually the cheapest. For context, exa is $5 / 1000.

deleted 1y ago

[deleted]