6 ms·

Internet Archive Scholar

446 points by nabla9 4y ago

schmudde 4y ago
I've already made my donation to IA this year but I might need to make another. Somehow it's the IA's job to fix problems that we all know are problems, sadly.
- wnkrshm 4y ago
  Wikipedia reminded me multiple times to donate to the Internet Archive this year.
  - nix23 4y ago
    Haha, i know what you mean...my mind works the same way ;)
  - xattt 4y ago
    Is this so Wikipedia can be archived by the IA?
    - wnkrshm 4y ago
      I always take the wikipedia donation drive as a reminder to donate to archive.org instead.
    - shepherdjerred 4y ago
      It's a joke. Hacker News doesn't like donating to Wikipedia; many choose to donate to Internet Archive instead.
      - tmtvl 4y ago
        HN doesn't mind donating to Wikipedia, they dislike donating to Wikimedia. That one letter is a big difference.
        shepherdjerred 4y ago
        Is there really a way to donate specifically to Wikipedia? I've never seen it in any of the many, many threads.
        tmtvl 4y ago
        As far as I know there isn't... that's the problem.
  - brookst 4y ago
    Same. Oh hey these scammers are asking for money again? Wait, I haven’t given to IA in a while.
  - password4321 4y ago
    Another chance to upvote the donation link to top thread on an IA story since a direct submission got swallowed by the dupe detector! They are doing such much amazing things. https://archive.org/donate https://archive.org/donate Your Donation Will Be Matched 2-to-1! [...] Right now, we have a 2-to-1 Matching Gift Campaign, tripling the impact of every donation. (from the home page)
- coldpie 4y ago
  If you're able/comfortable, please consider setting up a recurring donation. For long-term planning reasons, it's helpful for organizations to have a consistent recurring revenue stream that they can use to project assets further into the future. One-off donations are good, too! But if you're going to consistently send them money anyway, you may as well do it in a predictable manner to help their accounting.
MollyRealized 4y ago
This is amazing. Some of my grandfather's papers are on there. [1] He was a medical missionary in China and India. I doubt anyone in my family has had a chance to see these before. Quite a gift. I'm trying to get my grandmother's books onto there as well at some point. [1] - https://scholar.archive.org/search?q=%22Frederick+G.+Scovel%22 https://scholar.archive.org/search?q=%22Frederick+G.+Scovel%...
082349872349872 4y ago
cf https://gallica.bnf.fr/accueil/en/content/accueil-en?mode=desktop https://gallica.bnf.fr/accueil/en/content/accueil-en?mode=de...
veqq 4y ago
Some days, nearly half the links I click are dead so I've found myself relying on the waybackmachine more and more over the past few months. It's really shocking just how fast digital obsolescence reared its ugly head. Of course angelcites etc. were a clear early blow, but nowadays... I've started saving the html (including the css seems like too much overhead, and often it's incomplete or relies on downloads still - and screenshots are not searchable) of every interesting article I find online, downloading quite a few videos too with yt-dlp. I'd long copypasted all interesting comments into a txt file, but now it seems like data hoarding's the way to go - at least in moderation, focusing on things I'll actually refer back to. I remember 15 years ago, discovering pdf dumps on random sites like a kid in a candy store. Perhaps it'll be like that again, with people presenting museums of their favorite old pages.
- dspillett 4y ago
  > I've started saving I do similar. I've had https://github.com/ArchiveBox/ArchiveBox https://github.com/ArchiveBox/ArchiveBox bookmarked for a while as something to try better organise all that, but like a great many things I haven't go around to it yet.
  - circustaco 4y ago
    I use Raindrop for this. It’s a pretty great bookmark manager made by an indie dev, but it also can create archives of bookmarked pages. https://help.raindrop.io/backups#permanent-library https://help.raindrop.io/backups#permanent-library
    - patrickdward 4y ago
      I use Raindrop but didn't know abou that feature. Thanks.
    - dspillett 4y ago
      Thanks, I'll add that to the list of things to try out.
    - dspillett 4y ago
zozbot234 4y ago
Not from the IA, but see https://scholia.toolforge.org https://scholia.toolforge.org for an especially nice presentation of freely-available scholarly metadata.
- jboynyc 4y ago
  You may also be interested in OpenAlex.org which also uses wikidata (along with DOIs, ORCIDs, ISSNs and a few other standard identifiers) to classify publications.
sfusato 4y ago
archive.org is an alternative good internet as a giant library, as dreamed in early 90's: web archive, film archive, software archive, media archive... and now research papers archive.
maggu123 4y ago
riskable 4y ago
This seems like the type of thing that will become the search engine of first resort in the future as AI-generated propaganda and nonsense pollutes the spectrum of websites and search results.
- deleted 4y ago
  [deleted]
pabs3 4y ago
I wonder if this includes anything from Sci-Hub or are they unrelated.
- toomuchtodo 4y ago
  Artifacts that are of questionable legality due to copyright are archived but not made public for obvious reasons (typically referred to being “darked”).
photochemsyn 4y ago
After a little testing, this looks like a good information source, although the combination of Google Scholar and sci-hub is probably still the best option, i.e. I couldn't find anything with Internet Scholar that wasn't available with the other options, and the quality of results on searchs is somewhat higher with Google Scholar (this may be because Google Scholar utilizes citation count as a search parameter, which Internet Archive Scholar doesn't seem to do). Internet Archive is a great resource, however, it should get state funding as it provides a fundamentally important archival service. It's too bad it has to rely so much on private philanthropic donations (although state support comes with possible political interference, i.e. censorship of material that some politician doesn't like, maybe that's less of a problem with private donations, although then you could have some billionaire doing the same thing).
- bnewbold 4y ago
  The content in scholar.archive.org has been indexed in to Google Scholar (and other indices are likewise welcome to crawl the sitemap). There was some content "only" in scholar.archive.org, but now it should basically all be in Google Scholar. We haven't gotten around to describing this publicly, but it was an explicit decision and partnership between the organizations. Indeed scholar.archive.org does not currently use citation count in search rankings. We have a decent citation graph, which we are working to expose in scholar (it is visible in fatcat.wiki today). Would probably only ever use citation count as a weak boost in search rankings (eg, "any citations at all", "more than 25 citations" as boosts, nothing beyond that), don't want to create too strong a feedback loop influencing future citations. scholar.archive.org specifically was partially funded by the Mellon Foundation (and partially through donations and other service revenue). IA overall has diverse funding, including grants and service revenue from the USA (Library of Congress, IMLS, etc); other national governments (paid crawl services); foundation grants; universities and libraries (crawl, preservation, and digitization services); and of course general donations. The last category of course has the fewest strings and lets us pursue new projects which might be hard to get traditional funding for. Remember that the whole premise of web archiving was considered radical and quixotic at the beginning! (source: I work at IA on scholar)
- macrolime 4y ago
dang 4y ago
Related: Internet Archive Scholar - https://news.ycombinator.com/item?id=26419782 https://news.ycombinator.com/item?id=26419782 - March 2021 (3 comments) Internet Archive Scholar: Search Millions of Research Papers - https://news.ycombinator.com/item?id=26401568 https://news.ycombinator.com/item?id=26401568 - March 2021 (47 comments)
aaroninsf 4y ago
PSA the Internet Archive is a 501c3 non-profit (library!) and survives on donations and grants. A huge percentage of the operating budget is from small donors. The funding is preposterously small compared to other public-service public interest such as Wikipedia. A lot of us take it for granted and assume there is e.g. support from FAANG companies proportionate to the degree they lean on it. This is 100% NOT THE CASE. Please advocate for recurring institutional donations from your firm. The audience reading this has a lot of voice in a lot of organizations who could without a though sign up to make annual 10K, 100K, 1M donations... ...and essentially, none do. Please help change that!!! https://archive.org/donate/ https://archive.org/donate/
- doc_gunthrop 4y ago
  Anyone who uses amazon.com can set the Internet Archive as their preferred charity and shop using smile.amazon.com. A percentage of your purchases amount will go to the IA.
abriana_holaday 4y ago
Agreed

446 points by nabla9 4y ago

I've already made my donation to IA this year but I might need to make another. Somehow it's the IA's job to fix problems that we all know are problems, sadly.

wnkrshm 4y ago
Wikipedia reminded me multiple times to donate to the Internet Archive this year.
- nix23 4y ago
  Haha, i know what you mean...my mind works the same way ;)
- xattt 4y ago
  Is this so Wikipedia can be archived by the IA?
  - wnkrshm 4y ago
    I always take the wikipedia donation drive as a reminder to donate to archive.org instead.
  - shepherdjerred 4y ago
    It's a joke. Hacker News doesn't like donating to Wikipedia; many choose to donate to Internet Archive instead.
    - tmtvl 4y ago
      HN doesn't mind donating to Wikipedia, they dislike donating to Wikimedia. That one letter is a big difference.
      - shepherdjerred 4y ago
        Is there really a way to donate specifically to Wikipedia? I've never seen it in any of the many, many threads.
        tmtvl 4y ago
        As far as I know there isn't... that's the problem.
- brookst 4y ago
  Same. Oh hey these scammers are asking for money again? Wait, I haven’t given to IA in a while.
- password4321 4y ago
  Another chance to upvote the donation link to top thread on an IA story since a direct submission got swallowed by the dupe detector! They are doing such much amazing things. https://archive.org/donate https://archive.org/donate Your Donation Will Be Matched 2-to-1! [...] Right now, we have a 2-to-1 Matching Gift Campaign, tripling the impact of every donation. (from the home page)
coldpie 4y ago
If you're able/comfortable, please consider setting up a recurring donation. For long-term planning reasons, it's helpful for organizations to have a consistent recurring revenue stream that they can use to project assets further into the future. One-off donations are good, too! But if you're going to consistently send them money anyway, you may as well do it in a predictable manner to help their accounting.

MollyRealized 4y ago

This is amazing. Some of my grandfather's papers are on there. [1] He was a medical missionary in China and India. I doubt anyone in my family has had a chance to see these before. Quite a gift. I'm trying to get my grandmother's books onto there as well at some point. [1] - https://scholar.archive.org/search?q=%22Frederick+G.+Scovel%22 https://scholar.archive.org/search?q=%22Frederick+G.+Scovel%...

082349872349872 4y ago

cf https://gallica.bnf.fr/accueil/en/content/accueil-en?mode=desktop https://gallica.bnf.fr/accueil/en/content/accueil-en?mode=de...

veqq 4y ago

Some days, nearly half the links I click are dead so I've found myself relying on the waybackmachine more and more over the past few months. It's really shocking just how fast digital obsolescence reared its ugly head. Of course angelcites etc. were a clear early blow, but nowadays... I've started saving the html (including the css seems like too much overhead, and often it's incomplete or relies on downloads still - and screenshots are not searchable) of every interesting article I find online, downloading quite a few videos too with yt-dlp. I'd long copypasted all interesting comments into a txt file, but now it seems like data hoarding's the way to go - at least in moderation, focusing on things I'll actually refer back to. I remember 15 years ago, discovering pdf dumps on random sites like a kid in a candy store. Perhaps it'll be like that again, with people presenting museums of their favorite old pages.

dspillett 4y ago
> I've started saving I do similar. I've had https://github.com/ArchiveBox/ArchiveBox https://github.com/ArchiveBox/ArchiveBox bookmarked for a while as something to try better organise all that, but like a great many things I haven't go around to it yet.
- circustaco 4y ago
  I use Raindrop for this. It’s a pretty great bookmark manager made by an indie dev, but it also can create archives of bookmarked pages. https://help.raindrop.io/backups#permanent-library https://help.raindrop.io/backups#permanent-library
  - patrickdward 4y ago
    I use Raindrop but didn't know abou that feature. Thanks.
  - dspillett 4y ago
    Thanks, I'll add that to the list of things to try out.
  - dspillett 4y ago

zozbot234 4y ago

Not from the IA, but see https://scholia.toolforge.org https://scholia.toolforge.org for an especially nice presentation of freely-available scholarly metadata.

jboynyc 4y ago
You may also be interested in OpenAlex.org which also uses wikidata (along with DOIs, ORCIDs, ISSNs and a few other standard identifiers) to classify publications.

sfusato 4y ago

archive.org is an alternative good internet as a giant library, as dreamed in early 90's: web archive, film archive, software archive, media archive... and now research papers archive.

maggu123 4y ago

riskable 4y ago

This seems like the type of thing that will become the search engine of first resort in the future as AI-generated propaganda and nonsense pollutes the spectrum of websites and search results.

deleted 4y ago
[deleted]

pabs3 4y ago

I wonder if this includes anything from Sci-Hub or are they unrelated.

toomuchtodo 4y ago
Artifacts that are of questionable legality due to copyright are archived but not made public for obvious reasons (typically referred to being “darked”).

photochemsyn 4y ago

After a little testing, this looks like a good information source, although the combination of Google Scholar and sci-hub is probably still the best option, i.e. I couldn't find anything with Internet Scholar that wasn't available with the other options, and the quality of results on searchs is somewhat higher with Google Scholar (this may be because Google Scholar utilizes citation count as a search parameter, which Internet Archive Scholar doesn't seem to do). Internet Archive is a great resource, however, it should get state funding as it provides a fundamentally important archival service. It's too bad it has to rely so much on private philanthropic donations (although state support comes with possible political interference, i.e. censorship of material that some politician doesn't like, maybe that's less of a problem with private donations, although then you could have some billionaire doing the same thing).

bnewbold 4y ago
The content in scholar.archive.org has been indexed in to Google Scholar (and other indices are likewise welcome to crawl the sitemap). There was some content "only" in scholar.archive.org, but now it should basically all be in Google Scholar. We haven't gotten around to describing this publicly, but it was an explicit decision and partnership between the organizations. Indeed scholar.archive.org does not currently use citation count in search rankings. We have a decent citation graph, which we are working to expose in scholar (it is visible in fatcat.wiki today). Would probably only ever use citation count as a weak boost in search rankings (eg, "any citations at all", "more than 25 citations" as boosts, nothing beyond that), don't want to create too strong a feedback loop influencing future citations. scholar.archive.org specifically was partially funded by the Mellon Foundation (and partially through donations and other service revenue). IA overall has diverse funding, including grants and service revenue from the USA (Library of Congress, IMLS, etc); other national governments (paid crawl services); foundation grants; universities and libraries (crawl, preservation, and digitization services); and of course general donations. The last category of course has the fewest strings and lets us pursue new projects which might be hard to get traditional funding for. Remember that the whole premise of web archiving was considered radical and quixotic at the beginning! (source: I work at IA on scholar)
macrolime 4y ago

dang 4y ago

Related: Internet Archive Scholar - https://news.ycombinator.com/item?id=26419782 https://news.ycombinator.com/item?id=26419782 - March 2021 (3 comments) Internet Archive Scholar: Search Millions of Research Papers - https://news.ycombinator.com/item?id=26401568 https://news.ycombinator.com/item?id=26401568 - March 2021 (47 comments)

aaroninsf 4y ago

PSA the Internet Archive is a 501c3 non-profit (library!) and survives on donations and grants. A huge percentage of the operating budget is from small donors. The funding is preposterously small compared to other public-service public interest such as Wikipedia. A lot of us take it for granted and assume there is e.g. support from FAANG companies proportionate to the degree they lean on it. This is 100% NOT THE CASE. Please advocate for recurring institutional donations from your firm. The audience reading this has a lot of voice in a lot of organizations who could without a though sign up to make annual 10K, 100K, 1M donations... ...and essentially, none do. Please help change that!!! https://archive.org/donate/ https://archive.org/donate/

doc_gunthrop 4y ago
Anyone who uses amazon.com can set the Internet Archive as their preferred charity and shop using smile.amazon.com. A percentage of your purchases amount will go to the IA.

abriana_holaday 4y ago

Agreed