4 ms·

Whitehouse.gov petitions are blocked from search results

33 points by rquantz 9y ago

AaronFriel 9y ago
Of course they are, this administration has shown nothing but disdain for the general public, freedom of information, and a free press.
- praneshp 9y ago
  Please see samfriedman comment: https://news.ycombinator.com/item?id=15450853 https://news.ycombinator.com/item?id=15450853
samfriedman 9y ago
To anyone blaming the current administration, note that the robots.txt is identical before the election too: https://web.archive.org/web/20161101000359/https://petitions.whitehouse.gov/robots.txt https://web.archive.org/web/20161101000359/https://petitions...
- banku_brougham 9y ago
  thanks for pulling this up. i was going to be angry about the current administration, but now i'm angry about what amounts to standard practice in governance today. i guess its still better than the old days.
masukomi 9y ago
i would point out that robots.txt is optional you don't have to follow it. It would be easy enough for one of us to extract the text of each petition, with a simple spider, put it on a web site with links back to the original, and let google search that. The petitions are public documents for public consumption. Even if white house tried to sue it wouldn't be their content. it's the content of the person who created it. Otherwise they would be legally suggesting that they are petitioning themselves.... then again IANAL just a human capable of reasoning through things logically, which rarely has any bearing on lawsuits. ;)
gremlinsinc 9y ago
Someone should create a scraper/aggregator w/ links back and synopsis... So google does spider the content.
mrguyorama 9y ago
Isn't there a built in search page for these petitions? What good would it be to have these petitions indexed by google? To be honest, I don't really want petitions influenced by SEO
- rquantz 9y ago
  You don't think it would be good for people googling an issue to be able to find out that there's a relevant petition?

4 ms·

Whitehouse.gov petitions are blocked from search results

33 points by rquantz 9y ago

AaronFriel 9y ago
Of course they are, this administration has shown nothing but disdain for the general public, freedom of information, and a free press.
- praneshp 9y ago
  Please see samfriedman comment: https://news.ycombinator.com/item?id=15450853 https://news.ycombinator.com/item?id=15450853
samfriedman 9y ago
To anyone blaming the current administration, note that the robots.txt is identical before the election too: https://web.archive.org/web/20161101000359/https://petitions.whitehouse.gov/robots.txt https://web.archive.org/web/20161101000359/https://petitions...
- banku_brougham 9y ago
  thanks for pulling this up. i was going to be angry about the current administration, but now i'm angry about what amounts to standard practice in governance today. i guess its still better than the old days.
masukomi 9y ago
i would point out that robots.txt is optional you don't have to follow it. It would be easy enough for one of us to extract the text of each petition, with a simple spider, put it on a web site with links back to the original, and let google search that. The petitions are public documents for public consumption. Even if white house tried to sue it wouldn't be their content. it's the content of the person who created it. Otherwise they would be legally suggesting that they are petitioning themselves.... then again IANAL just a human capable of reasoning through things logically, which rarely has any bearing on lawsuits. ;)
gremlinsinc 9y ago
Someone should create a scraper/aggregator w/ links back and synopsis... So google does spider the content.
mrguyorama 9y ago
Isn't there a built in search page for these petitions? What good would it be to have these petitions indexed by google? To be honest, I don't really want petitions influenced by SEO
- rquantz 9y ago
  You don't think it would be good for people googling an issue to be able to find out that there's a relevant petition?