4 ms·

Out of curiosity, what manner of saving them would you suggest instead? I'm not sure it's possible in an automated way without technically violating their TOS.

by eli 11y ago

onedev 11y ago
You can automate taking screenshots of the rendered tweet the moment the API data comes in (e.g. before deletion)
- eli 11y ago
  Saving tweets that have been deleted violates the rules you agree to in order to get an API key. Also, they no longer have API keys. I think you could probably make an argument that it is fair use to save and display a tweet from a politician without violating anyone's copyright, but I'm not sure if it's possible to do it in an automated way without violating the TOS and possibly the law.
  - slg 11y ago
    I think scraping is probably the simplest but the ethics are just as questionable. Any method you try is probably going to be met with roadblocks from Twitter because you are inherently trying to use the site in a way that is against their rules. Like you said, there may be a fair use argument to be made, but that won't stop Twitter from at least trying to make it as hard as possible for you.
    - eli 11y ago
      I linked this in another comment: https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc. Scraping a site that doesn't want to be scraped is legally dangerous in the US.
      - slg 11y ago
        Which is where that fair use argument will come in to play. That is something that can only play out in court. Twitter has no responsibility to bend to fair use until a court makes them.
        eli 11y ago
        IANAL, but Fair Use is a statutory exemption to copyright law only. Not CFAA.
      - jayess 11y ago
        Case docket is here: https://www.unitedstatescourts.org/federal/cand/257395/ https://www.unitedstatescourts.org/federal/cand/257395/
  - onedev 11y ago
    They wouldn't be able to tell that it's automated though. You can probably do it with a single twitter account, without explicit API access. Just have your account follow all the politicians and screenshot the differences in your feed.
leni536 11y ago
Couldn't one just scrape? You don't have to use the API.
- jc4p 11y ago
  > without technically violating their TOS. Scraping is against most sites (and Twitter)'s ToS.
  - tedunangst 11y ago
    That's generally unenforceable unless you took affirmative action to agree to the terms. There's no proof the user even saw the terms if they're just scraping the site. Until you get a c&d letter, at which point it becomes harder to claim ignorance.
  - leni536 11y ago
    (I'm not a lawyer, IANAL) I don't get this. When did I officially agree to their ToS? I automatically agree on the first HTTP request? I think if I never had an account or never requested an API key then I didn't agree to it. Having said that there could be other legal issues to scraping (like copyright of posted tweets), but maybe ToS is not one of them.

Out of curiosity, what manner of saving them would you suggest instead? I'm not sure it's possible in an automated way without technically violating their TOS.

by eli 11y ago

Out of curiosity, what manner of saving them would you suggest instead? I'm not sure it's possible in an automated way without technically violating their TOS.

onedev 11y ago

You can automate taking screenshots of the rendered tweet the moment the API data comes in (e.g. before deletion)

eli 11y ago
Saving tweets that have been deleted violates the rules you agree to in order to get an API key. Also, they no longer have API keys. I think you could probably make an argument that it is fair use to save and display a tweet from a politician without violating anyone's copyright, but I'm not sure if it's possible to do it in an automated way without violating the TOS and possibly the law.
- slg 11y ago
  I think scraping is probably the simplest but the ethics are just as questionable. Any method you try is probably going to be met with roadblocks from Twitter because you are inherently trying to use the site in a way that is against their rules. Like you said, there may be a fair use argument to be made, but that won't stop Twitter from at least trying to make it as hard as possible for you.
  - eli 11y ago
    I linked this in another comment: https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc https://en.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc. Scraping a site that doesn't want to be scraped is legally dangerous in the US.
    - slg 11y ago
      Which is where that fair use argument will come in to play. That is something that can only play out in court. Twitter has no responsibility to bend to fair use until a court makes them.
      - eli 11y ago
        IANAL, but Fair Use is a statutory exemption to copyright law only. Not CFAA.
    - jayess 11y ago
      Case docket is here: https://www.unitedstatescourts.org/federal/cand/257395/ https://www.unitedstatescourts.org/federal/cand/257395/
- onedev 11y ago
  They wouldn't be able to tell that it's automated though. You can probably do it with a single twitter account, without explicit API access. Just have your account follow all the politicians and screenshot the differences in your feed.

leni536 11y ago

Couldn't one just scrape? You don't have to use the API.

jc4p 11y ago
> without technically violating their TOS. Scraping is against most sites (and Twitter)'s ToS.
- tedunangst 11y ago
  That's generally unenforceable unless you took affirmative action to agree to the terms. There's no proof the user even saw the terms if they're just scraping the site. Until you get a c&d letter, at which point it becomes harder to claim ignorance.
- leni536 11y ago
  (I'm not a lawyer, IANAL) I don't get this. When did I officially agree to their ToS? I automatically agree on the first HTTP request? I think if I never had an account or never requested an API key then I didn't agree to it. Having said that there could be other legal issues to scraping (like copyright of posted tweets), but maybe ToS is not one of them.