5 ms·

What makes processing searches so expensive? I see elsewhere in this forum that it’s $1/80 searches. I’m just curious what part of your infra is costing you so

What makes processing searches so expensive? I see elsewhere in this forum that it’s $1/80 searches. I’m just curious what part of your infra is costing you so much.

freediver 4y ago
Great question! Mostly paid APIs we use (probably 80%), rest is infrastructure. But I will counter with this. It costs $0.0125 to run a web search over the entire web, producing meaningful, relevant and rich results, and all that in less than 500ms, any time of day, from anywhere. That is 1.25 cents to do all of the above. And the value you can get from that one search can often be 1,000,000x more. That has to be one of the best deals in the universe. (and mind you, with other search engines that cost is still there, it is only paid by their customers - the advertisers)
- lumost 4y ago
  I suspect that your estimate of google search infrastructure costs may be high by 1-2 orders of magnitude. Have you considered what the cost would be if the external APIs were in-housed? As an early-stage startup, you likewise are paying extra for compute, and expedient but often inefficient systems. Is there a path to a <$5/mo unlimited plan? IIRC Google made less than $10 per year per user up until the mid 2010s.
  - freediver 4y ago
    You are probably right if we disregard the 20 years of cap-ex investments Google made into having that infrastructure. In order to in-house this we would probably need to own $500M-$1B worth of infrastructure at the very least (web is vast). Yes then the cost per search would be lower. Maybe it will sound strange, but I do not think of our goal as optimizing for growth and number of users. It is to optimize for sustainability and quality of product. Often these two incentives are opposite of one another. I see us as a small, boutique, premium search brand. I'd rather have us gain more users by improving the quality than by lowering the price.
    - gavinmckenzie 4y ago
      This strategy of not focusing on growth is key to my interests. For example, I’ve been a customer of Fastmail since they launched maybe 20 years ago. And I’ve got my family and many colleagues using Fastmail. It is a paid email service. Could I use any of the “free” email services like GMail (or back in the day, Yahoo)? Of course. But Fastmail does one thing and they do it well, and I don’t have to worry about them analyzing my emails and selling ads to me. I am willing to pay for that. Please stay focused. I just tried Kagi today and I’m incredibly surprised at how fantastic the search results are. I never thought I would pay for a search engine but here I am.
- Winsaucerer 4y ago
  Assuming you experience good growth in your business, is there room for you to reduce your dependency on external API's, while maintaining the good quality search results you seem to be generating? And to add to what others have been saying, I've been finding your results to be quite high quality, absolutely better than DDG and probably better than Google for me too. I subscribed immediately. The only thing I'm missing is I'm still getting slow result times from Australia. Just now, for a test search: Connected to: ASIA-SOUTHEAST Network latency: 501ms
  - freediver 4y ago
    We'd like to better understand what is going on. Would you be able to post here ? https://kagifeedback.org/d/183-kagi-slow-post-here https://kagifeedback.org/d/183-kagi-slow-post-here Recording a side by side video of DuckDuckGo results and Kagi results with network tab in browser open would be great, if you can.

What makes processing searches so expensive? I see elsewhere in this forum that it’s $1/80 searches. I’m just curious what part of your infra is costing you so much.

freediver 4y ago

Great question! Mostly paid APIs we use (probably 80%), rest is infrastructure. But I will counter with this. It costs $0.0125 to run a web search over the entire web, producing meaningful, relevant and rich results, and all that in less than 500ms, any time of day, from anywhere. That is 1.25 cents to do all of the above. And the value you can get from that one search can often be 1,000,000x more. That has to be one of the best deals in the universe. (and mind you, with other search engines that cost is still there, it is only paid by their customers - the advertisers)

lumost 4y ago
I suspect that your estimate of google search infrastructure costs may be high by 1-2 orders of magnitude. Have you considered what the cost would be if the external APIs were in-housed? As an early-stage startup, you likewise are paying extra for compute, and expedient but often inefficient systems. Is there a path to a <$5/mo unlimited plan? IIRC Google made less than $10 per year per user up until the mid 2010s.
- freediver 4y ago
  You are probably right if we disregard the 20 years of cap-ex investments Google made into having that infrastructure. In order to in-house this we would probably need to own $500M-$1B worth of infrastructure at the very least (web is vast). Yes then the cost per search would be lower. Maybe it will sound strange, but I do not think of our goal as optimizing for growth and number of users. It is to optimize for sustainability and quality of product. Often these two incentives are opposite of one another. I see us as a small, boutique, premium search brand. I'd rather have us gain more users by improving the quality than by lowering the price.
  - gavinmckenzie 4y ago
    This strategy of not focusing on growth is key to my interests. For example, I’ve been a customer of Fastmail since they launched maybe 20 years ago. And I’ve got my family and many colleagues using Fastmail. It is a paid email service. Could I use any of the “free” email services like GMail (or back in the day, Yahoo)? Of course. But Fastmail does one thing and they do it well, and I don’t have to worry about them analyzing my emails and selling ads to me. I am willing to pay for that. Please stay focused. I just tried Kagi today and I’m incredibly surprised at how fantastic the search results are. I never thought I would pay for a search engine but here I am.
Winsaucerer 4y ago
Assuming you experience good growth in your business, is there room for you to reduce your dependency on external API's, while maintaining the good quality search results you seem to be generating? And to add to what others have been saying, I've been finding your results to be quite high quality, absolutely better than DDG and probably better than Google for me too. I subscribed immediately. The only thing I'm missing is I'm still getting slow result times from Australia. Just now, for a test search: Connected to: ASIA-SOUTHEAST Network latency: 501ms
- freediver 4y ago
  We'd like to better understand what is going on. Would you be able to post here ? https://kagifeedback.org/d/183-kagi-slow-post-here https://kagifeedback.org/d/183-kagi-slow-post-here Recording a side by side video of DuckDuckGo results and Kagi results with network tab in browser open would be great, if you can.