5 ms·

Bypass Paywalls Clean for Chrome/Firefox

35 points by joker765 6y ago

joker765 6y ago
Repo moved to BitBucket: https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-clean https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl... https://bitbucket.org/magnolia1234/bypass-paywalls-fixed-clean https://bitbucket.org/magnolia1234/bypass-paywalls-fixed-cle...
joker765 6y ago
Repo moved to BitBucket (ff typo): https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-clean https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl... https://bitbucket.org/magnolia1234/bypass-paywalls-firefox-clean https://bitbucket.org/magnolia1234/bypass-paywalls-firefox-c...
joker765 6y ago
Refactored extension/add-on with lots of new sites, bug-fixes, add custom sites and update-notification. https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... (no google analytics) https://github.com/magnolia1234-new/bypass-paywalls-firefox-clean https://github.com/magnolia1234-new/bypass-paywalls-firefox-...
- Tepix 6y ago
  Has the link been updated? It's what the story is pointing to. Anyway, nice work.
  - joker765 6y ago
    Old account was suspended after complaints of iamadamdev of spamming his repo (while he copied commits), I guess.
- charlietran 6y ago
  How is this related to the existing bypass-paywalls extensions from https://github.com/iamadamdev https://github.com/iamadamdev?
  - joker765 6y ago
    Hard fork/mirror after very slow progress ... Lots of new sites and features.
diwu1989 6y ago
How does this work?
- wtallis 6y ago
  It's a variety of techniques depending on the site. See https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean/blob/master/background.js https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... and https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean/blob/master/contentScript.js https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... Techniques include blocking cookies, specific JS scripts, redirecting to a different URL, changing the user-agent to pretend to be Google's indexer, changing the referer to make it appear the user is coming from a Google search, and removing or hiding elements from the page that get in the way of reading the article.
andrenotgiant 6y ago
There is currently a really interesting paywall-bypass bug on NYTimes.com: - Take a paywall'ed URL: https://www.nytimes.com/2013/05/05/magazine/y-combinator-silicon-valleys-start-up-machine.html https://www.nytimes.com/2013/05/05/magazine/y-combinator-sil... - Add a dot after the domain: https://www.nytimes.com./2013/05/05/magazine/y-combinator-silicon-valleys-start-up-machine.html https://www.nytimes.com./2013/05/05/magazine/y-combinator-si... voila, no paywall. Would love to hear an inside story about why this bug appeared.
- dkdk8283 6y ago
  Unfortunately this bug will probably get fixed from your post
  - bestnameever 6y ago
    I really feel like it's just a matter of time where they require an account to read the articles and only provide snippets to those who are beyond there free limit.
- crankylinuxuser 6y ago
  Thats the difference between a domain name and a fully qualified domain name. If you've ever administered BIND, you'll note that you need to add a ending . to A records. Thats the FQDN. Now, our browsers link cookies to domain. And interestingly enough (example.com)!=(example.com.) That's why sites like nytimes.com. gets around the paywall, because you don't have a cookie history with that site.
- tipsysquid 6y ago
  I can't say for certain why this bug is exposed by NYTimes; however, I saw a comment on HN about trailing dots on domain names[1] being a browser implementation quirk. The author of that article suggests that NYTimes may have misconfigured their Apache VirtualHost. [1] http://www.dns-sd.org/TrailingDotsInDomainNames.html http://www.dns-sd.org/TrailingDotsInDomainNames.html
  - 6y ago
smitty1e 6y ago
There's so much content on the internet that, if I stub my toe on a paywall, I move on to the next link.
electriclove 6y ago
Did this get manually bumped off the homepage?
deleted 6y ago
[deleted]

35 points by joker765 6y ago

Repo moved to BitBucket: https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-clean https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl... https://bitbucket.org/magnolia1234/bypass-paywalls-fixed-clean https://bitbucket.org/magnolia1234/bypass-paywalls-fixed-cle...

joker765 6y ago

Repo moved to BitBucket (ff typo): https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-clean https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl... https://bitbucket.org/magnolia1234/bypass-paywalls-firefox-clean https://bitbucket.org/magnolia1234/bypass-paywalls-firefox-c...

joker765 6y ago

Refactored extension/add-on with lots of new sites, bug-fixes, add custom sites and update-notification. https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... (no google analytics) https://github.com/magnolia1234-new/bypass-paywalls-firefox-clean https://github.com/magnolia1234-new/bypass-paywalls-firefox-...

Tepix 6y ago
Has the link been updated? It's what the story is pointing to. Anyway, nice work.
- joker765 6y ago
  Old account was suspended after complaints of iamadamdev of spamming his repo (while he copied commits), I guess.
charlietran 6y ago
How is this related to the existing bypass-paywalls extensions from https://github.com/iamadamdev https://github.com/iamadamdev?
- joker765 6y ago
  Hard fork/mirror after very slow progress ... Lots of new sites and features.

diwu1989 6y ago

How does this work?

wtallis 6y ago
It's a variety of techniques depending on the site. See https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean/blob/master/background.js https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... and https://github.com/magnolia1234-new/bypass-paywalls-chrome-clean/blob/master/contentScript.js https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... Techniques include blocking cookies, specific JS scripts, redirecting to a different URL, changing the user-agent to pretend to be Google's indexer, changing the referer to make it appear the user is coming from a Google search, and removing or hiding elements from the page that get in the way of reading the article.

andrenotgiant 6y ago

There is currently a really interesting paywall-bypass bug on NYTimes.com: - Take a paywall'ed URL: https://www.nytimes.com/2013/05/05/magazine/y-combinator-silicon-valleys-start-up-machine.html https://www.nytimes.com/2013/05/05/magazine/y-combinator-sil... - Add a dot after the domain: https://www.nytimes.com./2013/05/05/magazine/y-combinator-silicon-valleys-start-up-machine.html https://www.nytimes.com./2013/05/05/magazine/y-combinator-si... voila, no paywall. Would love to hear an inside story about why this bug appeared.

dkdk8283 6y ago
Unfortunately this bug will probably get fixed from your post
- bestnameever 6y ago
  I really feel like it's just a matter of time where they require an account to read the articles and only provide snippets to those who are beyond there free limit.
crankylinuxuser 6y ago
Thats the difference between a domain name and a fully qualified domain name. If you've ever administered BIND, you'll note that you need to add a ending . to A records. Thats the FQDN. Now, our browsers link cookies to domain. And interestingly enough (example.com)!=(example.com.) That's why sites like nytimes.com. gets around the paywall, because you don't have a cookie history with that site.
tipsysquid 6y ago
I can't say for certain why this bug is exposed by NYTimes; however, I saw a comment on HN about trailing dots on domain names[1] being a browser implementation quirk. The author of that article suggests that NYTimes may have misconfigured their Apache VirtualHost. [1] http://www.dns-sd.org/TrailingDotsInDomainNames.html http://www.dns-sd.org/TrailingDotsInDomainNames.html
- 6y ago

smitty1e 6y ago

There's so much content on the internet that, if I stub my toe on a paywall, I move on to the next link.

electriclove 6y ago

Did this get manually bumped off the homepage?

deleted 6y ago

[deleted]