4 ms·

“Page had always wanted to digitize books. Way back in 1996, the student project that eventually became Google—a “crawler” that would ingest documents and rank

“Page had always wanted to digitize books. Way back in 1996, the student project that eventually became Google—a “crawler” that would ingest documents and rank them for relevance against a user’s query—was actually conceived as part of an effort “to develop the enabling technologies for a single, integrated and universal digital library.” The idea was that in the future, once all books were digitized, you’d be able to map the citations among them, see which books got cited the most, and use that data to give better search results to library patrons. But books still lived mostly on paper. Page and his research partner, Sergey Brin, developed their popularity-contest-by-citation idea using pages from the World Wide Web.“ Larry Page had some cool ideas… can’t imagine Books will ever be resurrected, unfortunately.

dekhn 2y ago
He really wanted to digitize all of them to provide reference and training data for early language models (well before LLMs, transformers, etc). He also had a plan (with George Church) to build enormous warehouses holding large-scale biology research infrastructure right next to google data centers. Because most biology research is done at locations that have reached their limit on computational/storage capacity. Larry had many good ideas but he struggled to get the majority of them off the ground. For example, when Trump was president and invited all the major tech leaders, Larry came with a plan to upgrade the US electrical system with long-range DC.
- shiroiushi 2y ago
  >Larry came with a plan to upgrade the US electrical system with long-range DC. I feel like some crucial detail is missing here. They already use HVDC for long-distance transmission lines, inside and outside of the US. Texas could benefit from it I suppose, but the US in general already uses it where appropriate AFAIK.
- pyrale 2y ago
  > Larry had many good ideas but he struggled to get the majority of them off the ground. For example, when Trump was president and invited all the major tech leaders, Larry came with a plan to upgrade the US electrical system with long-range DC. I fail to see how that would be a good idea. To me, it looks like some magnate in a completely unrelated industry, who is megalomaniac enough to believe that they can enter a completely unrelated industry and explain to experts how things ought to get done.
  - gosub100 2y ago
    He didn't originate the idea, it's well known that DC is more efficient but has other serious hurdles. It often takes innovative and influential people to break the status quo. E.g. Elon and EVs.
    - pyrale 2y ago
      > it's well known that DC is more efficient but has other serious hurdles. If it's well known but no one has done it, maybe there are reasons for it? Building long-distance lines is a very capital-intensive decision, and if the cost-benefit analysis was better, projects would be done. Looking by DOE's communication [1], we can see that the cost-benefit analysis doesn't look good now for a big network, but also that HVDC projects that actually make economic sense have been built for decades. Sure, sometimes innovative people make a big change, but what's the innovation there? [1]: https://www.energy.gov/oe/articles/connecting-country-hvdc https://www.energy.gov/oe/articles/connecting-country-hvdc
      - gosub100 2y ago
        There absolutely are HVDC transmission lines under the sea. And capital intensive, revolutionary projects are ripe for disruption from people who have already pulled off similar feats, hence why it would make sense for a Google founder to take interest.
        pyrale 2y ago
        > There absolutely are HVDC transmission lines under the sea. Not sure where you see me claiming the opposite? > And capital intensive, revolutionary projects are ripe for disruption from people who have already pulled off similar feats If Larry Page is looking to waste a few billions in cables, he's absolutely welcome to do it. One HVDC connection between France and the UK is private, for instance [1]. If he believes there's money to be made similarly in the US, he should go for it. That's not exactly the same offer as "Larry page has suggested the government should invest in expensive infrastructure for an industry he's never worked in". [1] https://en.wikipedia.org/wiki/ElecLink https://en.wikipedia.org/wiki/ElecLink
carlosjobim 2y ago
> The idea was that in the future, once all books were digitized, you’d be able to map the citations among them, see which books got cited the most, and use that data to give better search results to library patrons. You can do something similar to this already, by mapping which books are cited in Wikipedia articles. If you know how to do such a thing, because I don't.
- aspenmayer 2y ago
  Not specific to Wikipedia: https://aarontay.medium.com/3-new-tools-to-try-for-literature-mapping-connected-papers-inciteful-and-litmaps-a399f27622a https://aarontay.medium.com/3-new-tools-to-try-for-literatur... https://archive.is/Ul13s https://archive.is/Ul13s Specific to Wikipedia: Wikipedia Citations: Reproducible Citation Extraction from Multilingual Wikipedia [2024] https://arxiv.org/abs/2406.19291v1 https://arxiv.org/abs/2406.19291v1 https://doi.org/10.48550/arXiv.2406.19291 https://doi.org/10.48550/arXiv.2406.19291 > Wikipedia is an essential component of the open science ecosystem, yet it is poorly integrated with academic open science initiatives. Wikipedia Citations is a project that focuses on extracting and releasing comprehensive datasets of citations from Wikipedia. A total of 29.3 million citations were extracted from English Wikipedia in May 2020. Following this one-off research project, we designed a reproducible pipeline that can process any given Wikipedia dump in the cloud-based settings. To demonstrate its usability, we extracted 40.6 million citations in February 2023 and 44.7 million citations in February 2024. Furthermore, we equipped the pipeline with an adapted Wikipedia citation template translation module to process multilingual Wikipedia articles in 15 European languages so that they are parsed and mapped into a generic structured citation template. This paper presents our open-source software pipeline to retrieve, classify, and disambiguate citations on demand from a given Wikipedia dump. Prior work referenced in above abstract with some team overlap: Wikipedia citations: A comprehensive data set of citations with identifiers extracted from English Wikipedia [2021] https://direct.mit.edu/qss/article/2/1/1/97565/Wikipedia-citations-A-comprehensive-data-set-of https://direct.mit.edu/qss/article/2/1/1/97565/Wikipedia-cit... https://doi.org/10.1162/qss_a_00105 https://doi.org/10.1162/qss_a_00105 Datasets: A Comprehensive Dataset of Classified Citations with Identifiers from English Wikipedia (2024) https://zenodo.org/records/10782978 https://zenodo.org/records/10782978 https://doi.org/10.5281/zenodo.10782978 https://doi.org/10.5281/zenodo.10782978 A Comprehensive Dataset of Classified Citations with Identifiers from Multilingual Wikipedia (2024) https://zenodo.org/records/11210434 https://zenodo.org/records/11210434 https://doi.org/10.5281/zenodo.11210434 https://doi.org/10.5281/zenodo.11210434 Code (MIT License): https://github.com/albatros13/wikicite https://github.com/albatros13/wikicite https://github.com/albatros13/wikicite/tree/multilang https://github.com/albatros13/wikicite/tree/multilang Bonus links: https://www.mediawiki.org/wiki/Alternative_parsers https://www.mediawiki.org/wiki/Alternative_parsers https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wikipedias-citations-are-influencing-scholars-and-publishers/ https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wi...
lqstuart 2y ago
…and then they sold out to a Wall Street dickhead, and here we are

dekhn 2y ago

He really wanted to digitize all of them to provide reference and training data for early language models (well before LLMs, transformers, etc). He also had a plan (with George Church) to build enormous warehouses holding large-scale biology research infrastructure right next to google data centers. Because most biology research is done at locations that have reached their limit on computational/storage capacity. Larry had many good ideas but he struggled to get the majority of them off the ground. For example, when Trump was president and invited all the major tech leaders, Larry came with a plan to upgrade the US electrical system with long-range DC.

shiroiushi 2y ago
>Larry came with a plan to upgrade the US electrical system with long-range DC. I feel like some crucial detail is missing here. They already use HVDC for long-distance transmission lines, inside and outside of the US. Texas could benefit from it I suppose, but the US in general already uses it where appropriate AFAIK.
pyrale 2y ago
> Larry had many good ideas but he struggled to get the majority of them off the ground. For example, when Trump was president and invited all the major tech leaders, Larry came with a plan to upgrade the US electrical system with long-range DC. I fail to see how that would be a good idea. To me, it looks like some magnate in a completely unrelated industry, who is megalomaniac enough to believe that they can enter a completely unrelated industry and explain to experts how things ought to get done.
- gosub100 2y ago
  He didn't originate the idea, it's well known that DC is more efficient but has other serious hurdles. It often takes innovative and influential people to break the status quo. E.g. Elon and EVs.
  - pyrale 2y ago
    > it's well known that DC is more efficient but has other serious hurdles. If it's well known but no one has done it, maybe there are reasons for it? Building long-distance lines is a very capital-intensive decision, and if the cost-benefit analysis was better, projects would be done. Looking by DOE's communication [1], we can see that the cost-benefit analysis doesn't look good now for a big network, but also that HVDC projects that actually make economic sense have been built for decades. Sure, sometimes innovative people make a big change, but what's the innovation there? [1]: https://www.energy.gov/oe/articles/connecting-country-hvdc https://www.energy.gov/oe/articles/connecting-country-hvdc
    - gosub100 2y ago
      There absolutely are HVDC transmission lines under the sea. And capital intensive, revolutionary projects are ripe for disruption from people who have already pulled off similar feats, hence why it would make sense for a Google founder to take interest.
      - pyrale 2y ago
        > There absolutely are HVDC transmission lines under the sea. Not sure where you see me claiming the opposite? > And capital intensive, revolutionary projects are ripe for disruption from people who have already pulled off similar feats If Larry Page is looking to waste a few billions in cables, he's absolutely welcome to do it. One HVDC connection between France and the UK is private, for instance [1]. If he believes there's money to be made similarly in the US, he should go for it. That's not exactly the same offer as "Larry page has suggested the government should invest in expensive infrastructure for an industry he's never worked in". [1] https://en.wikipedia.org/wiki/ElecLink https://en.wikipedia.org/wiki/ElecLink

carlosjobim 2y ago

> The idea was that in the future, once all books were digitized, you’d be able to map the citations among them, see which books got cited the most, and use that data to give better search results to library patrons. You can do something similar to this already, by mapping which books are cited in Wikipedia articles. If you know how to do such a thing, because I don't.

aspenmayer 2y ago
Not specific to Wikipedia: https://aarontay.medium.com/3-new-tools-to-try-for-literature-mapping-connected-papers-inciteful-and-litmaps-a399f27622a https://aarontay.medium.com/3-new-tools-to-try-for-literatur... https://archive.is/Ul13s https://archive.is/Ul13s Specific to Wikipedia: Wikipedia Citations: Reproducible Citation Extraction from Multilingual Wikipedia [2024] https://arxiv.org/abs/2406.19291v1 https://arxiv.org/abs/2406.19291v1 https://doi.org/10.48550/arXiv.2406.19291 https://doi.org/10.48550/arXiv.2406.19291 > Wikipedia is an essential component of the open science ecosystem, yet it is poorly integrated with academic open science initiatives. Wikipedia Citations is a project that focuses on extracting and releasing comprehensive datasets of citations from Wikipedia. A total of 29.3 million citations were extracted from English Wikipedia in May 2020. Following this one-off research project, we designed a reproducible pipeline that can process any given Wikipedia dump in the cloud-based settings. To demonstrate its usability, we extracted 40.6 million citations in February 2023 and 44.7 million citations in February 2024. Furthermore, we equipped the pipeline with an adapted Wikipedia citation template translation module to process multilingual Wikipedia articles in 15 European languages so that they are parsed and mapped into a generic structured citation template. This paper presents our open-source software pipeline to retrieve, classify, and disambiguate citations on demand from a given Wikipedia dump. Prior work referenced in above abstract with some team overlap: Wikipedia citations: A comprehensive data set of citations with identifiers extracted from English Wikipedia [2021] https://direct.mit.edu/qss/article/2/1/1/97565/Wikipedia-citations-A-comprehensive-data-set-of https://direct.mit.edu/qss/article/2/1/1/97565/Wikipedia-cit... https://doi.org/10.1162/qss_a_00105 https://doi.org/10.1162/qss_a_00105 Datasets: A Comprehensive Dataset of Classified Citations with Identifiers from English Wikipedia (2024) https://zenodo.org/records/10782978 https://zenodo.org/records/10782978 https://doi.org/10.5281/zenodo.10782978 https://doi.org/10.5281/zenodo.10782978 A Comprehensive Dataset of Classified Citations with Identifiers from Multilingual Wikipedia (2024) https://zenodo.org/records/11210434 https://zenodo.org/records/11210434 https://doi.org/10.5281/zenodo.11210434 https://doi.org/10.5281/zenodo.11210434 Code (MIT License): https://github.com/albatros13/wikicite https://github.com/albatros13/wikicite https://github.com/albatros13/wikicite/tree/multilang https://github.com/albatros13/wikicite/tree/multilang Bonus links: https://www.mediawiki.org/wiki/Alternative_parsers https://www.mediawiki.org/wiki/Alternative_parsers https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wikipedias-citations-are-influencing-scholars-and-publishers/ https://scholarlykitchen.sspnet.org/2022/11/01/guest-post-wi...

lqstuart 2y ago

…and then they sold out to a Wall Street dickhead, and here we are