4 ms·

With search, there are actually 2 different products. The index itself is a textbook natural monopoly. There's no upside in having more than 1 web crawler as lo

With search, there are actually 2 different products. The index itself is a textbook natural monopoly. There's no upside in having more than 1 web crawler as long as it can be trusted to just crawl the web but there's plenty of downside in the form of increased bandwidth costs for web sites. The actual search rankings are where competition makes sense. So spin the Google search index off as a non-profit that makes its product available at a reasonable cost to anybody who wants to run a search engine. Then you spin off the rest of Google as separate companies. Android/Pixel, Google Ads/Analytics, Youtube, Chrome, Gmail/Docs/Drive and Maps/Cloud would all be separate businesses. Chrome would likely be best spunoff as a non-profit as well, operating an open source web browser project. Of course, if you're going to break up Google you also have to break up the rest of FAANG too or they're just going to step right into the space vacated by Google and probably buy up the Baby Googles[0]. Apple would probably need to be broken up into a hardware company, a software company and a services company. Amazon would need to be broken up into AWS, Amazon Shopping and Prime Video. You could arguably even break Amazon Shopping up into the online store and the logistics company. Microsoft would need to be broken up into Windows, Office, Xbox, Bing and Azure. Xbox would then itself need to be broken up to unwind all of its purchases of independent video game companies. If you're breaking up Xbox, you also have to do the same with Playstation (spinning them loose from Sony and unwinding their own acquisitions). It might also be a great idea to then do the same with major 3rd party video game companies like EA, basically unwinding all of their acquisitions. Meta would need to be broken up into Facebook, Oculus, Instagram, WhatsApp and Meta Ads. Once you start breaking up Google, you open up Pandora's box and basically have to keep going until you break up the entire tech sector. That's why it is so tricky to do it. Once you're done with tech, it'd be a good idea to look at other sectors such as health care, Hollywood and the news media where consolidation has been rampant over the past few decades. [0]: https://en.wikipedia.org/wiki/Regional_Bell_Operating_Company https://en.wikipedia.org/wiki/Regional_Bell_Operating_Compan...

cogman10 2y ago
> The index itself is a textbook natural monopoly. I would argue that because of this, the index itself should be something operated by the public rather than privately. This is especially true because it's within the best interests of the public to have an archive of the internet. The wayback machine currently supports that and it's on its last legs. Index the internet with a public search API, then let future search engines hit that an optimize based on internal algorithms to get better results.
- kajecounterhack 2y ago
  > the index itself should be something operated by the public rather than privately I love this but the act of indexing the web is both complex and ever-evolving (not to mention expensive), so the market needs to have natural incentives for the public crawler to continue being good at its job. Google is incentivized to have a great index because it feeds into its competitive advantage. If the index is a shared resource, what incentive does the indexing company have to keep improving it? And what guarantees that the indexing company's strategy doesn't end up being unfair to those affected by their websites getting indexed? All search companies would have to maintain secondary indices to augment the public one. There's probably some artificial structure needed to make open indexing viable.
  - cogman10 2y ago
    > what incentive does the indexing company have to keep improving it? A publicly owned asset doesn't need to be managed by a private entity. The incentive can be exactly the same incentive that the NHS and NASA operate on. It's a fun problem that you can make a career on making better that is ultimately a public good. Not everything needs to be operated with pure profits in mind. > And what guarantees that the indexing company's strategy doesn't end up being unfair to those affected by their websites getting indexed? Voting/lobbying/etc. Though I'd assume everyone that makes a website will want to be indexed.
    - ratorx 2y ago
      NHS is a pretty terrible example. Since it relies on public funding without a direct monetary profit, it is an easy target for defunding when the government needs to cut expenses. And lo and behold that is exactly what has happened. I’m not saying privatisation is better (US healthcare is the textbook counter-example), but just wanted to make the point that socialising something has demonstrably not been a good way to drive innovation and improvement in some cases.
    - kajecounterhack 2y ago
      > A publicly owned asset doesn't need to be managed by a private entity. The incentive can be exactly the same incentive that the NHS and NASA operate on. It's a fun problem that you can make a career on making better that is ultimately a public good. Not everything needs to be operated with pure profits in mind. Sure like there's open street maps for maps, there could be something like that for search indices. But I thought we were discussing the viability of making search indexing an independent, maybe even regulated natural monopoly as a way to democratize search and keep competition improving search itself. And I think the answer is no because search companies will find alpha from maintaining their own secondary indices and use it as a competitive advantage, and the 3rd party index will be (and if not immediately, then over time become) inferior to what the search companies use to differentiate themselves due to natural pressures.
warkdarrior 2y ago
> The index itself is a textbook natural monopoly. That is clearly not true. Marginalia has a different index than Kagi, which has a different index than Google. You are somewhat making the assumption that there is some perfect index that works for everyone, when the market shows there is a need for different indices for different users.
schott12521 2y ago
I can't begin to imagine what this world would look like, but I can keep dreaming... I think splitting up Maps from Google/Apple would make OpenStreetMaps see a lot more usage and improvements. Amazon Shopping should absolutely be split into an online store and a logistics company. I'd like to see something even more radical when splitting up Infrastructure (AWS / GCP / Azure), such as the Government offering Infrastructure similar to the Postal Service, effectively turning AWS / GCP / Azure into Fedex / UPS.

cogman10 2y ago

> The index itself is a textbook natural monopoly. I would argue that because of this, the index itself should be something operated by the public rather than privately. This is especially true because it's within the best interests of the public to have an archive of the internet. The wayback machine currently supports that and it's on its last legs. Index the internet with a public search API, then let future search engines hit that an optimize based on internal algorithms to get better results.

kajecounterhack 2y ago
> the index itself should be something operated by the public rather than privately I love this but the act of indexing the web is both complex and ever-evolving (not to mention expensive), so the market needs to have natural incentives for the public crawler to continue being good at its job. Google is incentivized to have a great index because it feeds into its competitive advantage. If the index is a shared resource, what incentive does the indexing company have to keep improving it? And what guarantees that the indexing company's strategy doesn't end up being unfair to those affected by their websites getting indexed? All search companies would have to maintain secondary indices to augment the public one. There's probably some artificial structure needed to make open indexing viable.
- cogman10 2y ago
  > what incentive does the indexing company have to keep improving it? A publicly owned asset doesn't need to be managed by a private entity. The incentive can be exactly the same incentive that the NHS and NASA operate on. It's a fun problem that you can make a career on making better that is ultimately a public good. Not everything needs to be operated with pure profits in mind. > And what guarantees that the indexing company's strategy doesn't end up being unfair to those affected by their websites getting indexed? Voting/lobbying/etc. Though I'd assume everyone that makes a website will want to be indexed.
  - ratorx 2y ago
    NHS is a pretty terrible example. Since it relies on public funding without a direct monetary profit, it is an easy target for defunding when the government needs to cut expenses. And lo and behold that is exactly what has happened. I’m not saying privatisation is better (US healthcare is the textbook counter-example), but just wanted to make the point that socialising something has demonstrably not been a good way to drive innovation and improvement in some cases.
  - kajecounterhack 2y ago
    > A publicly owned asset doesn't need to be managed by a private entity. The incentive can be exactly the same incentive that the NHS and NASA operate on. It's a fun problem that you can make a career on making better that is ultimately a public good. Not everything needs to be operated with pure profits in mind. Sure like there's open street maps for maps, there could be something like that for search indices. But I thought we were discussing the viability of making search indexing an independent, maybe even regulated natural monopoly as a way to democratize search and keep competition improving search itself. And I think the answer is no because search companies will find alpha from maintaining their own secondary indices and use it as a competitive advantage, and the 3rd party index will be (and if not immediately, then over time become) inferior to what the search companies use to differentiate themselves due to natural pressures.

warkdarrior 2y ago

> The index itself is a textbook natural monopoly. That is clearly not true. Marginalia has a different index than Kagi, which has a different index than Google. You are somewhat making the assumption that there is some perfect index that works for everyone, when the market shows there is a need for different indices for different users.

schott12521 2y ago

I can't begin to imagine what this world would look like, but I can keep dreaming... I think splitting up Maps from Google/Apple would make OpenStreetMaps see a lot more usage and improvements. Amazon Shopping should absolutely be split into an online store and a logistics company. I'd like to see something even more radical when splitting up Infrastructure (AWS / GCP / Azure), such as the Government offering Infrastructure similar to the Postal Service, effectively turning AWS / GCP / Azure into Fedex / UPS.