4 ms·
I'm not a researcher, but don't most researchers these days also upload their work to arXiv? Sure, it's not a journal - but in some fields (Machine Learning, M
by TrackerFF 1y ago
I'm not a researcher, but don't most researchers these days also upload their work to arXiv?
Sure, it's not a journal - but in some fields (Machine Learning, Math) it seems like everyone also uploads their stuff there. So if the models can crawls sites like arXiv, at least there's some decent stuff to be found.
- levocardia 1y agoNot outside of ML, physics, and math. Preprints are extremely rare in many (dare I say most) scientific fields, and of course many times you are interested in not the cutting edge work, but the foundational work in a field from the 60s, 70s, or 80s, all of which is locked behind a paywall. Or at least it's supposed to be, and corporate LLMs are not "allowed" to go poking around on sketchy Russian website for non-paywalled versions.
- qingcharles 1y agoI'm assuming many AI companies are probably scraping all the PDFs from the "shadow libraries" of the world that have done some of the work of liberating these papers from behind their paywalls. Obviously it's legally unsettled territory right now...
- jsemrau 1y agoProper research, especially the one contributed to conferences, is hard to get by and is usually managed by the conference organizers. Arxiv has some, but it's limited. It would be great if for a DeepSearch tool for ML, I could just use Arxiv as a source and have the Agent search this. But so far I have not found a working Arxiv tool that does this well.