7 ms·

There's always two things that these armchair quarterback software devs seem to miss: Scale and Money. Sure you can hack together a small Google clone using El

There's always two things that these armchair quarterback software devs seem to miss: Scale and Money. Sure you can hack together a small Google clone using Elasticsearch in a weekend. Can you index all the worlds information and serve the worlds search needs with your little MVP? 99.99% chance no, even if you were given a few hundred engineers to scale it. Even if you could do it technically, where are you coming up with the money to do so? That's not seed-round or even series A round money to do that. That's series D, hundred+ million dollar swipes of a VC's credit card just to get off the ground. Microsoft threw billions at it with Bing and still never became more than an also-ran. The only place that seems to have a real chance is DuckDuckGo, and I'd be very interested to see how their story has played out up to this point. Google benefited greatly from growing as the web was growing. Trying to index the world's information in 2020 is an astronomical task compared to indexing the world's information in 98, 99 era.

gustavo-fring 6y ago
I've been playing armchair QB for 20 years, my QBR is 91. Nobody is seriously saying index all of the world's information in a weekend. They're saying that the tech to index the web and otherwise has increased in the last 20 years, which enables you to do more. Google isn't even a great example, considering how intensive it is at scale. There are a lot of apps, especially with user created content, that don't have that issue at all. Your argument is the other end of I can build Google in a day, it dismisses criticism by making it absurd.
- kelnos 6y ago
  The "I could build it in a weekend" is certainly one extreme, but I think even people who say "I could build Google in 3 months" (or even 6 months, or a year), just don't get it. And yes, there are actually a decent amount of people who say, literally, "I could build that in a weekend". Not sure how many people say that about Google, specifically, but it comes up often enough. This post was also from 2016, and I do feel like it used to come up more often in years past than it does now.
  - heavenlyblue 6y ago
    Most of the people I know, who say “I could build that in a weekend” mean that they could build a product that solves 90% of the problems in that problem space in a weekend. And that’s true. The other 10% is what gives you competitive advantage as a product. The truth is that _if_ the market was empty at the time; then those 90% would give you enough runway to finish the rest of the 10% with a certain high enoufh probability.
    - a1369209993 6y ago
      I can't speak for anyone else, but as someone who has literally said that on several occasions, you've perfectly summarized ( 90% of :) ) what I generally mean.
mettamage 6y ago
> Scale and Money This is too simple of an answer. By these metrics Google+ would be a success. Also, the total funding amound of DDG is 13M (see Crunchbase). While that is serious money, that's not series D money. Moreover, there are counterexamples of companies that did build an app in a relatively short amount of time, kept a small team and made millions per employee. I don't know of many [1], but I think WuFoo is an interesting example, founded in 2006 while SurveyMonkey was founded in 1999. WuFoo exited for 35 million in 2011 with a team of 9, I think (there's a YC YouTube video about this somewhere). Disclaimer: I'm an armchair quarterback. [1] I don't know of many tech companies in general.
- astrea 6y ago
  Google+ isn't a fair counter-example. From a technical perspective, it very well could've been. The failings of Google+ was social.
- pheug 6y ago
  DDG is not in the index-all-the-web business. For the most part they are just serving Bing's results, that's the only reason why they operate with so little funding - it doesn't take much to operate a glorified proxy. Microsoft's footing the bills for the actual web search operation and I'm sure it's way above 13M.
- drevil-v2 6y ago
  WhatsApp is another example. I believe it was 13(?) employees, $16 billion exit and close to a billion users worldwide
  - mettamage 6y ago
    Yea, also pretty small scale-wise, from what I've heard.
- mrkeen 6y ago
  > > Scale and Money > This is too simple of an answer Writer implied necessity, not sufficiency.
3pt14159 6y ago
Bing was my secret weapon when doing data science contracts that were MVPs or side features. Take a couple million preselected strings they needed to match to and then run them through Bing, page through the results, and build up the corpus I needed out of something like mechanize / nokogiri / headless browser.[0] Then clean up the data the normal data science way then do whatever mathy stuff I needed to layer on for whatever app they needed. There you go mister client something that has billions of dollars of R&D behind it and cleaned up for your specific use-case. You want something better? Go off and hire a team of Phds and spend a couple years and 50x the money I charged you to get your RoC curves (or whatever) looking 10% or 15% better. Haven't done this in a couple years due to a startup and then after that randomly finding a client I liked working with so much I haven't needed to take on random jobs again, but I'm sure it would still work. [0] I had also written custom tools to make this easier / saner. Also inspector gadget + CSS selectors goes a long way too.
- ineedasername 6y ago
  That is not a scalable practice. You may have been able to do it, but if it became a too common a practice among developers making similar things then I'm sure Bing would have taken steps to limit such activity.
  - 3pt14159 6y ago
    That's why it was a secret weapon and not a "I told everyone how to do it" weapon. Also, lol, I spelled it "RoC curves" I clearly didn't proofread this.
    - dr_zoidberg 6y ago
      I'm pretty sure that's how you spell it: https://en.wikipedia.org/wiki/Receiver_operating_characteristic https://en.wikipedia.org/wiki/Receiver_operating_characteris...
      - 3pt14159 6y ago
        I mean, on a good day I capitalize the "O" in ROC.
jonas21 6y ago
> Microsoft threw billions at it with Bing and still never became more than an also-ran. The only place that seems to have a real chance is DuckDuckGo, and I'd be very interested to see how their story has played out up to this point. That's an interesting perspective when you consider that: 1. Bing has at least 5x the market share of DuckDuckGo in the US (and even higher worldwide) 2. DuckDuckGo uses Bing as its main source of search results.
- ascendantlogic 6y ago
  Anecdotally I just see a lot more people using DuckDuckGo as an alternative to Google. I will be the first to admit that I was clearly wrong in my assessment.
- ViViDboarder 6y ago
  > 2. DuckDuckGo uses Bing as its main source of search results. Is this still the case? I thought they now produce their own results.
  - jonas21 6y ago
    DDG is a little bit vague about it on their website, but their staff regularly mentions that the results are primarily from Bing on /r/duckduckgo, for example: https://www.reddit.com/r/duckduckgo/comments/g8wukz/duckduckgo_broken/fou0fzn/ https://www.reddit.com/r/duckduckgo/comments/g8wukz/duckduck...
Axsuul 6y ago
...and time. DuckDuckGo is 11 years old and just getting started.
kanzenryu2 6y ago
I've seen people saying Google have effectively given up indexing the entire web. Pages older than 10 years don't tend to get indexed anymore, for example. Similar for search in gmail.
tonyedgecombe 6y ago
I've always thought there is little point trying to attack a company like Google directly. Rather you would be better trying to carve out a niche. For instance why isn't there a search engine dedicated to what programmers need.
divbzero 6y ago
In addition to all the above, there is money in the “long tails” of search. At scale, even the 0.01% use case will have thousands of hits per day. This is already true at DuckDuckGo’s current scale of ~50M daily traffic [1] and even more so for giants like Google or Baidu. [1]: https://duckduckgo.com/traffic https://duckduckgo.com/traffic

gustavo-fring 6y ago

I've been playing armchair QB for 20 years, my QBR is 91. Nobody is seriously saying index all of the world's information in a weekend. They're saying that the tech to index the web and otherwise has increased in the last 20 years, which enables you to do more. Google isn't even a great example, considering how intensive it is at scale. There are a lot of apps, especially with user created content, that don't have that issue at all. Your argument is the other end of I can build Google in a day, it dismisses criticism by making it absurd.

kelnos 6y ago
The "I could build it in a weekend" is certainly one extreme, but I think even people who say "I could build Google in 3 months" (or even 6 months, or a year), just don't get it. And yes, there are actually a decent amount of people who say, literally, "I could build that in a weekend". Not sure how many people say that about Google, specifically, but it comes up often enough. This post was also from 2016, and I do feel like it used to come up more often in years past than it does now.
- heavenlyblue 6y ago
  Most of the people I know, who say “I could build that in a weekend” mean that they could build a product that solves 90% of the problems in that problem space in a weekend. And that’s true. The other 10% is what gives you competitive advantage as a product. The truth is that _if_ the market was empty at the time; then those 90% would give you enough runway to finish the rest of the 10% with a certain high enoufh probability.
  - a1369209993 6y ago
    I can't speak for anyone else, but as someone who has literally said that on several occasions, you've perfectly summarized ( 90% of :) ) what I generally mean.

mettamage 6y ago

> Scale and Money This is too simple of an answer. By these metrics Google+ would be a success. Also, the total funding amound of DDG is 13M (see Crunchbase). While that is serious money, that's not series D money. Moreover, there are counterexamples of companies that did build an app in a relatively short amount of time, kept a small team and made millions per employee. I don't know of many [1], but I think WuFoo is an interesting example, founded in 2006 while SurveyMonkey was founded in 1999. WuFoo exited for 35 million in 2011 with a team of 9, I think (there's a YC YouTube video about this somewhere). Disclaimer: I'm an armchair quarterback. [1] I don't know of many tech companies in general.

astrea 6y ago
Google+ isn't a fair counter-example. From a technical perspective, it very well could've been. The failings of Google+ was social.
pheug 6y ago
DDG is not in the index-all-the-web business. For the most part they are just serving Bing's results, that's the only reason why they operate with so little funding - it doesn't take much to operate a glorified proxy. Microsoft's footing the bills for the actual web search operation and I'm sure it's way above 13M.
drevil-v2 6y ago
WhatsApp is another example. I believe it was 13(?) employees, $16 billion exit and close to a billion users worldwide
- mettamage 6y ago
  Yea, also pretty small scale-wise, from what I've heard.
mrkeen 6y ago
> > Scale and Money > This is too simple of an answer Writer implied necessity, not sufficiency.

3pt14159 6y ago

Bing was my secret weapon when doing data science contracts that were MVPs or side features. Take a couple million preselected strings they needed to match to and then run them through Bing, page through the results, and build up the corpus I needed out of something like mechanize / nokogiri / headless browser.[0] Then clean up the data the normal data science way then do whatever mathy stuff I needed to layer on for whatever app they needed. There you go mister client something that has billions of dollars of R&D behind it and cleaned up for your specific use-case. You want something better? Go off and hire a team of Phds and spend a couple years and 50x the money I charged you to get your RoC curves (or whatever) looking 10% or 15% better. Haven't done this in a couple years due to a startup and then after that randomly finding a client I liked working with so much I haven't needed to take on random jobs again, but I'm sure it would still work. [0] I had also written custom tools to make this easier / saner. Also inspector gadget + CSS selectors goes a long way too.

ineedasername 6y ago
That is not a scalable practice. You may have been able to do it, but if it became a too common a practice among developers making similar things then I'm sure Bing would have taken steps to limit such activity.
- 3pt14159 6y ago
  That's why it was a secret weapon and not a "I told everyone how to do it" weapon. Also, lol, I spelled it "RoC curves" I clearly didn't proofread this.
  - dr_zoidberg 6y ago
    I'm pretty sure that's how you spell it: https://en.wikipedia.org/wiki/Receiver_operating_characteristic https://en.wikipedia.org/wiki/Receiver_operating_characteris...
    - 3pt14159 6y ago
      I mean, on a good day I capitalize the "O" in ROC.

jonas21 6y ago

> Microsoft threw billions at it with Bing and still never became more than an also-ran. The only place that seems to have a real chance is DuckDuckGo, and I'd be very interested to see how their story has played out up to this point. That's an interesting perspective when you consider that: 1. Bing has at least 5x the market share of DuckDuckGo in the US (and even higher worldwide) 2. DuckDuckGo uses Bing as its main source of search results.

ascendantlogic 6y ago
Anecdotally I just see a lot more people using DuckDuckGo as an alternative to Google. I will be the first to admit that I was clearly wrong in my assessment.
ViViDboarder 6y ago
> 2. DuckDuckGo uses Bing as its main source of search results. Is this still the case? I thought they now produce their own results.
- jonas21 6y ago
  DDG is a little bit vague about it on their website, but their staff regularly mentions that the results are primarily from Bing on /r/duckduckgo, for example: https://www.reddit.com/r/duckduckgo/comments/g8wukz/duckduckgo_broken/fou0fzn/ https://www.reddit.com/r/duckduckgo/comments/g8wukz/duckduck...

Axsuul 6y ago

...and time. DuckDuckGo is 11 years old and just getting started.

kanzenryu2 6y ago

I've seen people saying Google have effectively given up indexing the entire web. Pages older than 10 years don't tend to get indexed anymore, for example. Similar for search in gmail.

tonyedgecombe 6y ago

I've always thought there is little point trying to attack a company like Google directly. Rather you would be better trying to carve out a niche. For instance why isn't there a search engine dedicated to what programmers need.

divbzero 6y ago

In addition to all the above, there is money in the “long tails” of search. At scale, even the 0.01% use case will have thousands of hits per day. This is already true at DuckDuckGo’s current scale of ~50M daily traffic [1] and even more so for giants like Google or Baidu. [1]: https://duckduckgo.com/traffic https://duckduckgo.com/traffic