3 ms·

Your father's books are real treasures. Were it my library, they would have a place of honor. I want to put your post in the context of yesterday's HN front-pa

Your father's books are real treasures. Were it my library, they would have a place of honor. I want to put your post in the context of yesterday's HN front-page post lambasting the EU for trying to build a better search engine. The bulk of the comments suggested that a government effort could never be as good as a commercial effort. https://news.ycombinator.com/item?id=32915263#32916945 https://news.ycombinator.com/item?id=32915263#32916945 Your post is a strong counter-argument. All of the points you mention on the massive decline in quality of web content are due to the web being driven by commercial efforts. Likewise, comments on this current HN front-page post "Despite faster broadband every year, web pages don't load any faster" https://news.ycombinator.com/item?id=32945858 https://news.ycombinator.com/item?id=32945858 also seem to explain the poor state of the web as being due to commercialization of everything (even the comments about the need for cookie banners-- the logic behind the cookie consent is to regulate the commercial collection of user data). Google started as a DARPA project, and was a great engine while it stayed true to that ethos. It was the need to commercialize it, thus setting perverse business incentives, which has destroyed it. Your post praises libraries. Which are seldom commercial ventures. The economics are simple. I don't know why this even continues to be a debate on HN. - A socially created entity (corporation, government, "charitable" organization, ...) needs money to function. - Money comes from capturing a portion of the value created by the organization. - The most efficient organizational structure depends on the relationship between value creation and value capture. Thus: if the product/service generates immediate and focused value, the value capture can be directly linked to the product and a business is optimal. Think: a hamburger. If the product/service generates long-term and diffuse value, the value capture also needs to be diffused, i.e. taxes. Thus a government. Think: the road network which allows the raw materials and the customers to get to the hamburger store. I leave the case for charitable orgs as an exercise for the reader :) disclaimer: strongly pro-business, have founded 2 personally, assisted several others.

Eleison23 4y ago
>Google started as a DARPA project, and was a great engine while it stayed true to that ethos. It was the need to commercialize it, thus setting perverse business incentives, which has destroyed it. Google started as a what now? This is an interesting thing to say during a discussion of access to information. I'd be happy to read your explanation of Google's "DARPA roots", and your citations to sources explaining how that came about and how they were "destroyed" when they "strayed" from DARPA... that should be a fascinating read.
- mellavora 4y ago
  It is indeed a fascinating read. Here you go! https://qz.com/1145669/googles-true-origin-partly-lies-in-cia-and-nsa-research-grants-for-mass-surveillance/ https://qz.com/1145669/googles-true-origin-partly-lies-in-ci... >A second grant—the DARPA-NSF grant most closely associated with Google’s origin—was part of a coordinated effort to build a massive digital library using the internet as its backbone. Both grants funded research by two graduate students who were making rapid advances in web-page ranking, as well as tracking (and making sense of) user queries: future Google cofounders Sergey Brin and Larry Page. >The research by Brin and Page under these grants became the heart of Google: people using search functions to find precisely what they wanted inside a very large data set.
  - softcactus 4y ago
    I always wondered how their initial python web scraper was fast enough to index the entire internet on that old hardware, even given the small size of the internet at the time. I guess the answer is that they had a local backup at Stanford. Thanks for sharing!
- mellavora 4y ago
  You also asked 'how they were "destroyed"' I refer you to some interesting discussion here: Google Search is Dying https://news.ycombinator.com/item?id=30347719 https://news.ycombinator.com/item?id=30347719 1561 points Every Google result now looks like an ad https://news.ycombinator.com/item?id=22107823 https://news.ycombinator.com/item?id=22107823 972 commments Google no longer producing high quality search results in significant categories https://news.ycombinator.com/item?id=29772136 https://news.ycombinator.com/item?id=29772136 1275 comments
talideon 4y ago
Uh, DARPA had nothing to do with the founding of Google. Brin and Page had NSF funding though Brin's graduate fellowship and the Digital Library Initiative, and that was before the official founding of the company. Reference: https://beta.nsf.gov/news/origins-google https://beta.nsf.gov/news/origins-google
- mellavora 4y ago
  You may (or may not) be right about DARPA, but you assert they were government funded. And my main point is that government (funding) is the economically optimal approach for services which produce diffuse value. See also the comment from marginella_nu in the post I linked to. Marginella is building a fantastic alternative search engine https://search.marginalia.nu/ https://search.marginalia.nu/ Their view: "Arguably the biggest most unsolved problem in search is how to make a profit" i.e., capturing the value produced. kagi https://kagi.com/ https://kagi.com/ attempts to do this with a paid tier. I hope it works for them, great product and really responsive team. Bing tries to capture value by collecting the Microsoft tax; not exactly government-level, but on those lines.

Eleison23 4y ago

>Google started as a DARPA project, and was a great engine while it stayed true to that ethos. It was the need to commercialize it, thus setting perverse business incentives, which has destroyed it. Google started as a what now? This is an interesting thing to say during a discussion of access to information. I'd be happy to read your explanation of Google's "DARPA roots", and your citations to sources explaining how that came about and how they were "destroyed" when they "strayed" from DARPA... that should be a fascinating read.

mellavora 4y ago
It is indeed a fascinating read. Here you go! https://qz.com/1145669/googles-true-origin-partly-lies-in-cia-and-nsa-research-grants-for-mass-surveillance/ https://qz.com/1145669/googles-true-origin-partly-lies-in-ci... >A second grant—the DARPA-NSF grant most closely associated with Google’s origin—was part of a coordinated effort to build a massive digital library using the internet as its backbone. Both grants funded research by two graduate students who were making rapid advances in web-page ranking, as well as tracking (and making sense of) user queries: future Google cofounders Sergey Brin and Larry Page. >The research by Brin and Page under these grants became the heart of Google: people using search functions to find precisely what they wanted inside a very large data set.
- softcactus 4y ago
  I always wondered how their initial python web scraper was fast enough to index the entire internet on that old hardware, even given the small size of the internet at the time. I guess the answer is that they had a local backup at Stanford. Thanks for sharing!
mellavora 4y ago
You also asked 'how they were "destroyed"' I refer you to some interesting discussion here: Google Search is Dying https://news.ycombinator.com/item?id=30347719 https://news.ycombinator.com/item?id=30347719 1561 points Every Google result now looks like an ad https://news.ycombinator.com/item?id=22107823 https://news.ycombinator.com/item?id=22107823 972 commments Google no longer producing high quality search results in significant categories https://news.ycombinator.com/item?id=29772136 https://news.ycombinator.com/item?id=29772136 1275 comments

talideon 4y ago

Uh, DARPA had nothing to do with the founding of Google. Brin and Page had NSF funding though Brin's graduate fellowship and the Digital Library Initiative, and that was before the official founding of the company. Reference: https://beta.nsf.gov/news/origins-google https://beta.nsf.gov/news/origins-google

mellavora 4y ago
You may (or may not) be right about DARPA, but you assert they were government funded. And my main point is that government (funding) is the economically optimal approach for services which produce diffuse value. See also the comment from marginella_nu in the post I linked to. Marginella is building a fantastic alternative search engine https://search.marginalia.nu/ https://search.marginalia.nu/ Their view: "Arguably the biggest most unsolved problem in search is how to make a profit" i.e., capturing the value produced. kagi https://kagi.com/ https://kagi.com/ attempts to do this with a paid tier. I hope it works for them, great product and really responsive team. Bing tries to capture value by collecting the Microsoft tax; not exactly government-level, but on those lines.