Unpaywall is very helpful! However, even for direct PDF links, publishing platforms will often do things like check for a session cookie; if you don't have the

Unpaywall is very helpful! However, even for direct PDF links, publishing platforms will often do things like check for a session cookie; if you don't have the correct cookie you get bounced back to the landing page, where you need find and follow another link. This isn't super complicated to work around (persist a cookie jar, use a headless browser, etc), but it doesn't work out-of-the-box with our crawlers, the same way crawling youtube doesn't work out-of-the-box so we rely on the youtube-dl community.