6 ms·

Code to block unwanted News Genius annotations

46 points by runandcode 11y ago

colinbartlett 11y ago
This is pretty clever, thanks to the author for releasing it. Personally, I might rather just add the script tag to my site's layout rather than introduce another gem dependency. But maybe if you have lots of sites to maintain, this could be preferred?
runandcode 11y ago
Hey colinbartlett, OP here. Definitely a good point. Interestingly, I didn't set out intending to use JS to solve this problem. I wanted to use Rack to intercept the request and check the headers and proceed from there. However, when I was playing around with it in Request Bin, I noticed that Genius actually modifies the URL so that there is no reference to them anywhere in the headers. This is generally poor etiquette and forced me to modify my approach. Thanks for your comment!
- sbierwagen 11y ago
  Click the "reply" link on a comment to reply to that comment.
gkop 11y ago
Folks blocking, curious your motivations? (I can think of a few but am curious about the specific issues with Genius vs. the platonic ideal of a universal web annotation layer)
- anoonmoose 11y ago
  I'd have to imagine it's the same as the list of motivations one would provide for disabling comments, which I've seen happen on plenty of sites.
  - gkop 11y ago
    Motivations for disabling comments vary widely, for one thing. For another, Genius is a company with a history of controversy. I can imagine some sites being ok with universal annotations but averse to Genius. It'd be useful to know how Genius is going about things wrong, because I dream of someday having a universal annotation layer that doesn't provoke widespread blocking.
  - chatmasta 11y ago
    You mean like how CNN disables comments so that people are unable to call them out on their bullshit? So they can post fearmongering articles with little basis in reality, backed up by quotes from unnamed anonymous "intelligence sources?" So they can continue to spread misinformation on behalf of the US government without anyone questioning it? The mainstream media is a complete poison to society. It needs to be dismantled and ripped apart. It's a dark, dark villain that needs to be exposed to the sunlight and shown for the monster that it is. That would be easier if comments were allowed on "publications" like CNN, but unfortunately they're disabled. Personally, I am fully in support of federated "overlay" commenting systems like news genius, because they rightfully remove control/moderation over the comments from the publisher, and place it into the hands of a "trusted 3rd party" (genius).
- siegecraft 11y ago
  They probably don't want hostile/parasitic services profiting off their content for free, even though that seems to be the business plan of every other startup. If Genius wants to act in good faith they should let people opt-out of annotations on their sites / pages. Also, it's just another walled garden for content from a company with past ethically questionable business practices.
MaxfordAndSons 11y ago
What is the motivation behind this? It strikes me as something like an anti-ad-blocker tool, except instead of serving to protect your sites ad revenue it's just trying to suppress free discourse about your content through a third party tool that happens to be exposed to this kind of suppression, but isn't actually invasive, and is only used by people who opt-in to it anyway. Basically it seems to me like a tool for people to protect their ability to successfully disseminate misinformation. Which, I guess is your prerogative and all, but I personally find disheartening. Having an effective universal annotation platform in wide use across the web could go a long way towards improving the SNR for all users.
- sehr 11y ago
  You'd be able to make a blog post without annotations accusing you of disseminating misinformation and suppressing free thought, for one. I think you're severely overrating the quality of Internet comments too, and with these annotations there isn't even a report feature available to help clean them up. Genius.
  - MaxfordAndSons 11y ago
    > I think you're severely overrating the quality of Internet comments Maybe so, but I think perhaps you're underrating the desire by state and corporate actors for tools to suppress free discourse. > there isn't even a report feature available No, there definitely is a "Report Abuse" button. But, there clearly isn't much in the way of a global reputation/meta-moderation system, except what's carried over from the original Genius site, which doesn't exactly share the same editorial values as an (my) ideal annotation platform. The managing editor being from Gawker and going around posting snarky/superficially critical comments on serious pieces of writing is pretty emblematic of this. Which is a shame. Since it's related, and I'm a fan of their work, here's another annotation platform, https://hypothes.is/ https://hypothes.is/, whose developers seem much more serious about creating a robust reputation system.
    - runandcode 11y ago
      The Report Abuse button seems like it was just recently added, in the past day. It was not present when the tool was originally released.
tkone 11y ago
I wrote some javascript you can stick on your page that blocks all attempts for genius to inject shit into your DOM. https://github.com/toddself/genius-blocker https://github.com/toddself/genius-blocker
- runandcode 11y ago
  Hey tkone, Thanks for sharing. The gem works in a similar way, with a small JS script.
simonw 11y ago
In case you were wondering how it works, this library is a rack middleware wrapper around this JavaScript: <script> var annotated = window.location.href.indexOf("genius.it/") != -1; if (annotated) { window.location.href = window.location.href.replace("genius.it/", "") } </script>
- web007 11y ago
  Leftpad all over again.
  - runandcode 11y ago
    Hey, OP here. I originally intended to leverage Rack all the way through by inspecting headers. However, much to my chagrin, I discovered that Genius does not reveal its identity in the headers, so I had to resort to JS instead to accomplish my goal.
  - ebola1717 11y ago
    In this situation, genius will probably try to circumvent a bunch of these techniques. You could imagine this library adds more techniques to fight genius, genius adds work arounds, the library adds different techniques, and it goes back and forth. For users of the library, they just have to update the gem to stay ahead.
mchahn 11y ago
To me a big motivation is to keep control of the content on your page. Also, isn't it a copyright violation?
- Denzel 11y ago
  IANAL, but I absolutely believe there's the possibility to raise a copyright violation suit. There have been cases brought against Google in the past for caching webpages. This is why google/bing web crawlers abide by robots.txt. NewsGenius needs to at least 1) identify their crawler, and 2) provide a way to opt-out. Otherwise, it doesn't look good for them. They're not being a very good citizen of the Web.
donohoe 11y ago
See also this one by the talented Vijith Assar: https://github.com/vijithassar/genius-defender https://github.com/vijithassar/genius-defender Unlike the others (which prevent hosting on genius.it) this one actually prevents annotations via plugin etc by injecting invisible characters at random. Its a WP plugin but the method could be adapted. Unsure if there are SEO/other implications. More info: http://www.vijithassar.com/2641/how-to-block-genius-annotations http://www.vijithassar.com/2641/how-to-block-genius-annotati...
j_s 11y ago
Going a bit meta here: Is there a standard library to use for monitoring 3rd-party interactions users bring to a website? Typically this would mean browser add-ons and extensions, but I wouldn't be surprised to discover other things (like headers injected by cellular companies, or who knows what from malware infections). Such a tool would be useful when troubleshooting various 'works on my machine' scenarios.
gergles 11y ago
I guess Genius doesn't care about the rampant copyright violation involved with the "web annotator"? Using genius.it/ creates derived works, and it does so almost certainly without approval of the copyright owner. I guess they will argue fair use, but hostile 'annotations' definitely reduce the value of the work, they reuse the entire work, and they use the work for commercial gain. I wouldn't consider this a slam dunk. I actually really dislike this concept of "we're going to wrap your website and mark it up with other stuff to make it look like you're hosting this content here". Sure, people should be able to talk wherever they want, but keep that in a popup, or at worst a frame - don't highlight text to suggest that the author has done that, and don't load your own elements in their page.
- chatmasta 11y ago
  Thanks for the interesting, non-inflammatory comment. You raise a good point. The fact that they proxy the content on their servers definitely puts them in a more vulnerable position than if this were, for example, a strictly client-side chrome extension that fetched annotation metadata and re-rendered the page accordingly. In the purely client-side case, the full article would not pass through their servers, only bits and pieces of it. It would be easier to argue fair-use if you're only storing parts of the articles, and comments/highlights, on a remote server, instead of proxying the entire content. (What if the article is behind a paywall, for example? Then genius.it effectively becomes a circumvention proxy.)
dcole2929 11y ago
Didn't even know this was a thing but after taking a gander at the first example of this on the News Genius (http://news.genius.com/ http://news.genius.com/) website I'm not surprised people want to get rid of it. I read the 2nd article, a buzz feed production about Trump's campaign adviser, and honestly I'm highly disappointed at the quality of the annotations. Even more so when I found out that the person doing said annotations is the Managing News Editor at Genius. Buzz feed is by no means a bastion of journalistic integrity but the superficial comments this editor left are a joke. If she was actually the editor for this article it might make sense to complain about wording choices but as I understand it Genius Annotations are meant to explain and expand upon things in the article. Here, the person Genius felt most qualified to head this whole effort, uses annotations like a permanent twitter stream of her personal opinion of the article. She complains about not expanding upon allegations but the article does. Not always but in a lot of cases they can't. No one is going on the record against a man working for the likely republican nominee. They can't afford to burn bridges because they still need access to Trump. All in all if that is the quality of annotations created by Genius' Managing New Editor this is a feature that needs to be taken out back.
koolba 11y ago
I'm no JS expert[1] but this doesn't look correct: # https://github.com/marlabrizel/genius-blocker/blob/master/lib/genius/blocker.rb#L11 var annotated = window.location.href.indexOf("genius.it/") != -1; if (annotated) { window.location.href = window.location.href.replace("genius.it/", "") } Wouldn't this match any instance of "genius.it" throughout the URL, not just at the beginning? For example it'd match "https://example.com/blog/how-to-block-genius.it" https://example.com/blog/how-to-block-genius.it" and redirect the user to a (probably) non-existent link with that string removed. [1]: I just play one on HN.
- runandcode 11y ago
  Hey koolba, OP here. Thanks for the code review - it's always good to see suggestions on how to improve things. If you have ideas on how to make the code better, please feel free to submit a pull request. Thanks for sharing your feedback!
pbarnes_1 11y ago
If Genius were actually acting in good faith, they'd allow some sort of 'robots.txt' like functionality, which would solve this whole thing in a nice way. But that's unlikely.
- siegecraft 11y ago
  It would be nice if they made it even easier for users on hosted services who can't necessarily change robots.txt so you could opt out on a per-page basis. But I don't think anyone expects Genius to act in good faith.
tzs 11y ago
I do not understand how genius.it is legal. Based on what I read about it before I actually saw it in action, I had guessed that they were showing the original site in a frame served from the original site with their annotations outside that frame. (That may or may not be legal. I'm not sure if it has been settled whether or not displaying someone's page served from their site in a frame served from your site might be a violation of the public display copyright right). However, after looking at the genius.it page that was linked in another comment for part of this very thread, and also looking at the genius.it page for my own website, that's not the case. They appear to have made a copy of the base page itself and are serving that themselves. They don't seem to copy images and stylesheets--those are still being served from the original server. In the copy of the page they serve, they modify the links contained therein to be genius.it links. That's making a derivative work. I don't think Genius could argue successfully that this use is fair use. They are a commercial site, and they are copying the entire page from the original site.
- michaelmrose 11y ago
  Wouldn't it be extremely easy for this to happen in the users browser wherein you have no right nor power and just fetch annotations based on the current url? If it doesn't already work that way it would be trivial to do so.

46 points by runandcode 11y ago

This is pretty clever, thanks to the author for releasing it. Personally, I might rather just add the script tag to my site's layout rather than introduce another gem dependency. But maybe if you have lots of sites to maintain, this could be preferred?

runandcode 11y ago

Hey colinbartlett, OP here. Definitely a good point. Interestingly, I didn't set out intending to use JS to solve this problem. I wanted to use Rack to intercept the request and check the headers and proceed from there. However, when I was playing around with it in Request Bin, I noticed that Genius actually modifies the URL so that there is no reference to them anywhere in the headers. This is generally poor etiquette and forced me to modify my approach. Thanks for your comment!

sbierwagen 11y ago
Click the "reply" link on a comment to reply to that comment.

gkop 11y ago

Folks blocking, curious your motivations? (I can think of a few but am curious about the specific issues with Genius vs. the platonic ideal of a universal web annotation layer)

anoonmoose 11y ago
I'd have to imagine it's the same as the list of motivations one would provide for disabling comments, which I've seen happen on plenty of sites.
- gkop 11y ago
  Motivations for disabling comments vary widely, for one thing. For another, Genius is a company with a history of controversy. I can imagine some sites being ok with universal annotations but averse to Genius. It'd be useful to know how Genius is going about things wrong, because I dream of someday having a universal annotation layer that doesn't provoke widespread blocking.
- chatmasta 11y ago
  You mean like how CNN disables comments so that people are unable to call them out on their bullshit? So they can post fearmongering articles with little basis in reality, backed up by quotes from unnamed anonymous "intelligence sources?" So they can continue to spread misinformation on behalf of the US government without anyone questioning it? The mainstream media is a complete poison to society. It needs to be dismantled and ripped apart. It's a dark, dark villain that needs to be exposed to the sunlight and shown for the monster that it is. That would be easier if comments were allowed on "publications" like CNN, but unfortunately they're disabled. Personally, I am fully in support of federated "overlay" commenting systems like news genius, because they rightfully remove control/moderation over the comments from the publisher, and place it into the hands of a "trusted 3rd party" (genius).
siegecraft 11y ago
They probably don't want hostile/parasitic services profiting off their content for free, even though that seems to be the business plan of every other startup. If Genius wants to act in good faith they should let people opt-out of annotations on their sites / pages. Also, it's just another walled garden for content from a company with past ethically questionable business practices.

MaxfordAndSons 11y ago

What is the motivation behind this? It strikes me as something like an anti-ad-blocker tool, except instead of serving to protect your sites ad revenue it's just trying to suppress free discourse about your content through a third party tool that happens to be exposed to this kind of suppression, but isn't actually invasive, and is only used by people who opt-in to it anyway. Basically it seems to me like a tool for people to protect their ability to successfully disseminate misinformation. Which, I guess is your prerogative and all, but I personally find disheartening. Having an effective universal annotation platform in wide use across the web could go a long way towards improving the SNR for all users.

sehr 11y ago
You'd be able to make a blog post without annotations accusing you of disseminating misinformation and suppressing free thought, for one. I think you're severely overrating the quality of Internet comments too, and with these annotations there isn't even a report feature available to help clean them up. Genius.
- MaxfordAndSons 11y ago
  > I think you're severely overrating the quality of Internet comments Maybe so, but I think perhaps you're underrating the desire by state and corporate actors for tools to suppress free discourse. > there isn't even a report feature available No, there definitely is a "Report Abuse" button. But, there clearly isn't much in the way of a global reputation/meta-moderation system, except what's carried over from the original Genius site, which doesn't exactly share the same editorial values as an (my) ideal annotation platform. The managing editor being from Gawker and going around posting snarky/superficially critical comments on serious pieces of writing is pretty emblematic of this. Which is a shame. Since it's related, and I'm a fan of their work, here's another annotation platform, https://hypothes.is/ https://hypothes.is/, whose developers seem much more serious about creating a robust reputation system.
  - runandcode 11y ago
    The Report Abuse button seems like it was just recently added, in the past day. It was not present when the tool was originally released.

tkone 11y ago

I wrote some javascript you can stick on your page that blocks all attempts for genius to inject shit into your DOM. https://github.com/toddself/genius-blocker https://github.com/toddself/genius-blocker

runandcode 11y ago
Hey tkone, Thanks for sharing. The gem works in a similar way, with a small JS script.

simonw 11y ago

In case you were wondering how it works, this library is a rack middleware wrapper around this JavaScript: <script> var annotated = window.location.href.indexOf("genius.it/") != -1; if (annotated) { window.location.href = window.location.href.replace("genius.it/", "") } </script>

web007 11y ago
Leftpad all over again.
- runandcode 11y ago
  Hey, OP here. I originally intended to leverage Rack all the way through by inspecting headers. However, much to my chagrin, I discovered that Genius does not reveal its identity in the headers, so I had to resort to JS instead to accomplish my goal.
- ebola1717 11y ago
  In this situation, genius will probably try to circumvent a bunch of these techniques. You could imagine this library adds more techniques to fight genius, genius adds work arounds, the library adds different techniques, and it goes back and forth. For users of the library, they just have to update the gem to stay ahead.

mchahn 11y ago

To me a big motivation is to keep control of the content on your page. Also, isn't it a copyright violation?

Denzel 11y ago
IANAL, but I absolutely believe there's the possibility to raise a copyright violation suit. There have been cases brought against Google in the past for caching webpages. This is why google/bing web crawlers abide by robots.txt. NewsGenius needs to at least 1) identify their crawler, and 2) provide a way to opt-out. Otherwise, it doesn't look good for them. They're not being a very good citizen of the Web.

donohoe 11y ago

See also this one by the talented Vijith Assar: https://github.com/vijithassar/genius-defender https://github.com/vijithassar/genius-defender Unlike the others (which prevent hosting on genius.it) this one actually prevents annotations via plugin etc by injecting invisible characters at random. Its a WP plugin but the method could be adapted. Unsure if there are SEO/other implications. More info: http://www.vijithassar.com/2641/how-to-block-genius-annotations http://www.vijithassar.com/2641/how-to-block-genius-annotati...

j_s 11y ago

Going a bit meta here: Is there a standard library to use for monitoring 3rd-party interactions users bring to a website? Typically this would mean browser add-ons and extensions, but I wouldn't be surprised to discover other things (like headers injected by cellular companies, or who knows what from malware infections). Such a tool would be useful when troubleshooting various 'works on my machine' scenarios.

gergles 11y ago

I guess Genius doesn't care about the rampant copyright violation involved with the "web annotator"? Using genius.it/ creates derived works, and it does so almost certainly without approval of the copyright owner. I guess they will argue fair use, but hostile 'annotations' definitely reduce the value of the work, they reuse the entire work, and they use the work for commercial gain. I wouldn't consider this a slam dunk. I actually really dislike this concept of "we're going to wrap your website and mark it up with other stuff to make it look like you're hosting this content here". Sure, people should be able to talk wherever they want, but keep that in a popup, or at worst a frame - don't highlight text to suggest that the author has done that, and don't load your own elements in their page.

chatmasta 11y ago
Thanks for the interesting, non-inflammatory comment. You raise a good point. The fact that they proxy the content on their servers definitely puts them in a more vulnerable position than if this were, for example, a strictly client-side chrome extension that fetched annotation metadata and re-rendered the page accordingly. In the purely client-side case, the full article would not pass through their servers, only bits and pieces of it. It would be easier to argue fair-use if you're only storing parts of the articles, and comments/highlights, on a remote server, instead of proxying the entire content. (What if the article is behind a paywall, for example? Then genius.it effectively becomes a circumvention proxy.)

dcole2929 11y ago

Didn't even know this was a thing but after taking a gander at the first example of this on the News Genius (http://news.genius.com/ http://news.genius.com/) website I'm not surprised people want to get rid of it. I read the 2nd article, a buzz feed production about Trump's campaign adviser, and honestly I'm highly disappointed at the quality of the annotations. Even more so when I found out that the person doing said annotations is the Managing News Editor at Genius. Buzz feed is by no means a bastion of journalistic integrity but the superficial comments this editor left are a joke. If she was actually the editor for this article it might make sense to complain about wording choices but as I understand it Genius Annotations are meant to explain and expand upon things in the article. Here, the person Genius felt most qualified to head this whole effort, uses annotations like a permanent twitter stream of her personal opinion of the article. She complains about not expanding upon allegations but the article does. Not always but in a lot of cases they can't. No one is going on the record against a man working for the likely republican nominee. They can't afford to burn bridges because they still need access to Trump. All in all if that is the quality of annotations created by Genius' Managing New Editor this is a feature that needs to be taken out back.

koolba 11y ago

I'm no JS expert[1] but this doesn't look correct: # https://github.com/marlabrizel/genius-blocker/blob/master/lib/genius/blocker.rb#L11 var annotated = window.location.href.indexOf("genius.it/") != -1; if (annotated) { window.location.href = window.location.href.replace("genius.it/", "") } Wouldn't this match any instance of "genius.it" throughout the URL, not just at the beginning? For example it'd match "https://example.com/blog/how-to-block-genius.it" https://example.com/blog/how-to-block-genius.it" and redirect the user to a (probably) non-existent link with that string removed. [1]: I just play one on HN.

runandcode 11y ago
Hey koolba, OP here. Thanks for the code review - it's always good to see suggestions on how to improve things. If you have ideas on how to make the code better, please feel free to submit a pull request. Thanks for sharing your feedback!

pbarnes_1 11y ago

If Genius were actually acting in good faith, they'd allow some sort of 'robots.txt' like functionality, which would solve this whole thing in a nice way. But that's unlikely.

siegecraft 11y ago
It would be nice if they made it even easier for users on hosted services who can't necessarily change robots.txt so you could opt out on a per-page basis. But I don't think anyone expects Genius to act in good faith.

tzs 11y ago

I do not understand how genius.it is legal. Based on what I read about it before I actually saw it in action, I had guessed that they were showing the original site in a frame served from the original site with their annotations outside that frame. (That may or may not be legal. I'm not sure if it has been settled whether or not displaying someone's page served from their site in a frame served from your site might be a violation of the public display copyright right). However, after looking at the genius.it page that was linked in another comment for part of this very thread, and also looking at the genius.it page for my own website, that's not the case. They appear to have made a copy of the base page itself and are serving that themselves. They don't seem to copy images and stylesheets--those are still being served from the original server. In the copy of the page they serve, they modify the links contained therein to be genius.it links. That's making a derivative work. I don't think Genius could argue successfully that this use is fair use. They are a commercial site, and they are copying the entire page from the original site.

michaelmrose 11y ago
Wouldn't it be extremely easy for this to happen in the users browser wherein you have no right nor power and just fetch annotations based on the current url? If it doesn't already work that way it would be trivial to do so.