5 ms·

"Twitter has sold billions of archived tweets believed to have vanished forever. A privacy row has erupted as hundreds of companies queue up to purchase users’

"Twitter has sold billions of archived tweets believed to have vanished forever. A privacy row has erupted as hundreds of companies queue up to purchase users’ personal information from the new database." Is there actually any information in this? What do they mean by believed to be vanished forever? According to the BBC new article [1] "private accounts and tweets that have been deleted will not be indexed by the site." Historically Twitter have been pretty good about ensuring, in their licences at least, things like deleted tweets are deleted even from external archives. It's what makes compiling a database of tweets - even for research purposes - quite difficult. [1] http://www.bbc.co.uk/news/technology-17178022 http://www.bbc.co.uk/news/technology-17178022

saurik 15y ago
Normal people believe that information that is nearly impossibly difficult to find is as good as gone forever. (This is mostly as these same "normal people" also often don't understand that website scraping can be automated; a sad misapplication of almost-reasonable cost heuristics to a system they don't understand.) However, on Twitter there is an extra wrinkle: neither the site nor the API allow you to go back more than 800 tweets for any view, whether it be "search results matching X" or "tweets that mention me" or "tweets posted by user Y" (the one exception being "your own tweets", where you can go back 3200). This means that if you tweet a lot, it is impossible for another user to go back very far into your history. Even you may not be able to go back that far in your own history. As an example: if you send 20 tweets a day, your history will be inaccessible to you within 6 months, and inaccessible to other users within 6 weeks. Given that this information is seemingly just gone--inaccessible, unable to be found from the user interface of any client, or Twitter's website, or searches on Twitter--while you and I know that they didn't delete it (in fact, Twitter even implies as much in their documentation somewhere: that the 800/3200 are only "temporary" limits), normal people are going to consider that data "vanished forever".
- warp 15y ago
  If you know the id of a tweet you can still access it I believe. https://api.twitter.com/1/statuses/show.json?id=20 https://api.twitter.com/1/statuses/show.json?id=20 still works, which allows you to construct https://twitter.com/#!/jack/status/20 https://twitter.com/#!/jack/status/20 which also still works. So, it's only searches which don't go that far back.
  - saurik 15y ago
    Right. You can also find a lot of old tweets using Google (which is, of course, hit or miss). As I said, you (a general you of "people on HN", not just the person I responded to) and I know that it isn't deleted. You and I know all sorts of things about websites that the normal person does not, which means we honestly can (and do) come off as if we know deep hacker magic secrets for demonstrating what is, to us, rather obvious things. When we design websites (and privacy policies...) we need to remember this. Remember: for a normal person, those somewhat scary-looking URLs are likely opaque. To the extent that people spend any time pulling them apart, it turns into a game of "guess the really long number, by hand, typing attempts into the URL bar of their web browser". They are unlikely even to figure out that it is monotonically increasing (which may or may not even help given the volume of the site); and, even if they did, for them the probability of guessing one of those numbers would still be effectively zero.
simonw 15y ago
I imagine the "believed to have vanished forever" is based on the common misconception that because Twitter's search only goes back about a week, they don't save older tweets anywhere.

saurik 15y ago

Normal people believe that information that is nearly impossibly difficult to find is as good as gone forever. (This is mostly as these same "normal people" also often don't understand that website scraping can be automated; a sad misapplication of almost-reasonable cost heuristics to a system they don't understand.) However, on Twitter there is an extra wrinkle: neither the site nor the API allow you to go back more than 800 tweets for any view, whether it be "search results matching X" or "tweets that mention me" or "tweets posted by user Y" (the one exception being "your own tweets", where you can go back 3200). This means that if you tweet a lot, it is impossible for another user to go back very far into your history. Even you may not be able to go back that far in your own history. As an example: if you send 20 tweets a day, your history will be inaccessible to you within 6 months, and inaccessible to other users within 6 weeks. Given that this information is seemingly just gone--inaccessible, unable to be found from the user interface of any client, or Twitter's website, or searches on Twitter--while you and I know that they didn't delete it (in fact, Twitter even implies as much in their documentation somewhere: that the 800/3200 are only "temporary" limits), normal people are going to consider that data "vanished forever".

warp 15y ago
If you know the id of a tweet you can still access it I believe. https://api.twitter.com/1/statuses/show.json?id=20 https://api.twitter.com/1/statuses/show.json?id=20 still works, which allows you to construct https://twitter.com/#!/jack/status/20 https://twitter.com/#!/jack/status/20 which also still works. So, it's only searches which don't go that far back.
- saurik 15y ago
  Right. You can also find a lot of old tweets using Google (which is, of course, hit or miss). As I said, you (a general you of "people on HN", not just the person I responded to) and I know that it isn't deleted. You and I know all sorts of things about websites that the normal person does not, which means we honestly can (and do) come off as if we know deep hacker magic secrets for demonstrating what is, to us, rather obvious things. When we design websites (and privacy policies...) we need to remember this. Remember: for a normal person, those somewhat scary-looking URLs are likely opaque. To the extent that people spend any time pulling them apart, it turns into a game of "guess the really long number, by hand, typing attempts into the URL bar of their web browser". They are unlikely even to figure out that it is monotonically increasing (which may or may not even help given the volume of the site); and, even if they did, for them the probability of guessing one of those numbers would still be effectively zero.

simonw 15y ago

I imagine the "believed to have vanished forever" is based on the common misconception that because Twitter's search only goes back about a week, they don't save older tweets anywhere.