8 ms·

Even years later, Twitter doesn’t delete direct messages

130 points by RobertSmith 8y ago

maxxxxx 8y ago
Isn't it pretty safe to assume that nothing will ever get really deleted? I wonder if the GDPR will change this.
- nightfly 8y ago
  My first thought when people bring up deleting things is backup tapes. Sure it might or might not be "deleted" from the live data set, but it's almost always gonna live on in backups for probably forever.
  - NullPrefix 8y ago
    GDPR affects backup tapes too
    - kradroy 8y ago
      It does, but not like you'd think. If the backup is dormant, the right-to-be-forgotten portions can remain there. Once it comes out of cold storage, those RTBF requests have to be processed over the restored data. Although, in most cases, the dormant backups will expire and be deleted as per a data retention plan.
      - msh 8y ago
        That does not seem to be entirely correct: https://blog.quantum.com/backup-administrators-the-1-advice-to-deal-with-gdpr-and-the-right-of-erasure/#.XGfzF6TnvDu https://blog.quantum.com/backup-administrators-the-1-advice-...
        guitarbill 8y ago
        Which part isn't correct? What op described sounds like a good and easy way to do it (as long as you have sane retention periods) and doesn't seem to conflict with the article you linked in any way (in fact, they agree).
        msh 8y ago
        In addition you have to tell the customer how long you retain your backups (and only retain them for a reasonable time) and only restore for purely technical reasons.
      - fh973 8y ago
        This is usually solved by encrypting and throwing away the key or seed to effectively delete. Tape is no excuse to violating data retention policies.
  - porpoisely 8y ago
    I agree. I find it hard to believe that anyone is going to transport all the physical tapes from cold storage and look through all the physical tapes. My guess is that the data is removed on production and recovery sites. The data will be left on tape since they will most likely never see the light of day again.
    - fh973 8y ago
      This is usually solved by encrypting and throwing away the key or seed to effectively delete.
      - code_duck 8y ago
        Doesn’t that just effectively delete it until it is common to have sufficient computing power to crack the encryption?
- brootstrap 8y ago
  If people are using a real database, likely there is just a boolean flag for that record getting flipped to True/False to signify deletion. Unless you use mongo, then your shit is probably gone already and you dont even know it.
  - wbronitsky 8y ago
    Wait what? Why are you randomly crapping on Mongo? You can use Mongo in exactly the same way as you described.
    - pizza 8y ago
      It's hacker news, randomly crapping on technology nobody brought up is what hackers do (/s)
    - thanatos_dem 8y ago
      Because crapping on Mongo is “in” right now. Years ago it used to be “in” to crap on it for its deaign choices, especially its tendency to lose data (seems to be what the parent comment is referencing; see [1] for a summary of some issues from 2013). More recently it’s “in” for their business decisions, with the relicensing and the rise of API-compatible alternatives pushing some users away. But yeah, really no reason to mock them in this discussion, apart from going for internet cool points from an audience that may also be critical of Mongo. [1] https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb
  - talaketu 8y ago
    Real database supports real erasure.
ddebernardy 8y ago
It's a bit disingenuous to expect Twitter or any other social media to actually delete deleted data, except perhaps through a GDPR request or similar. Plus, if your account gets hacked and deleted, or if you delete the account and rejoin, you might be very happy that they didn't actually delete anything and are able to restore your account. (I'm not a fan of social networks or what they do with our data, but the database administrator in me tends to side with them on that front. Keeping data around is a lesser evil than losing it.)
- spudlyo 8y ago
  Your instincts are correct. I happen to know that DMs at Twitter are (and most likely still) stored in a sharded MySQL store[0]. Deleted DMs and tweets are "tombstoned", and not actually deleted via a DELETE, which has pretty terrible performance characteristics in MySQL [0]: http://highscalability.com/blog/2011/12/19/how-twitter-stores-250-million-tweets-a-day-using-mysql.html http://highscalability.com/blog/2011/12/19/how-twitter-store...
  - Operyl 8y ago
    I wonder what they use to generate IDs! /s Man that comment popped up a bunch in that blog post, lol.
- wavesounds 8y ago
  It's not disingenuous at all. Snapchat deletes all messages, photos and videos by default, either immediately after their received or within 24 hours.
- talaketu 8y ago
  That's a very wacky take. A general privacy principle is that if the user requests the service provider to delete the user's data, then the service provider should comply, unless there are legal injunctions. It's arguable about user expectation for direct messages on twitter, given there is a sender and receiver and Twitter in the middle, but what is not arguable is that if both users delete the message or their accounts, then the messages should not be retained by the Twitter.
B-Con 8y ago
A lot of the article focuses on DMs sent from eventually deleted/suspend accounts being accessible from the receiving account. When you send a message to another account, it's in their account now. Of course it's not going to disappear along with your account. This had been the canonical behavior of messaging clients for forever. Email obviously, but also IM, chat find, etc. (Trying to remember about BBSs.) Once you press send it's not yours anymore. This should 100% be the expectation of users anywhere they go. Once you press send, do not count on being able to withdraw it under any circumstances. Does this genuinely surprise people or is it a slow news day at TechCrunch?
- nikanj 8y ago
  The same went for letters too, back when those were still being written.
  - zapzupnz 8y ago
    With the difference being that nobody had a copy of your letters, just the recipient.
    - chiefalchemist 8y ago
      Not to sound like an ahole but I presume Twitter would argue it's __their__ message (in the sense that it's on their platform). with the sender and recipient having access to it. Once you hit submit you've forfeited "direct ownership" of the message.
      - mdhshjdjsh 8y ago
        AIM (AOL Instant Messenger) had a similar policy, which, I think, if I remember correctly, was a defining moment for AIM, and once again, for AOL. The EULA for AIM was such that your messages became AOL’s intellectual property. Not only that you could not sue AOL Time Warner for possibly scripting and producing a movie and thus profiting from one of your instant message conversations or chat transcripts, but that maybe they could sue you for trying to do the same, and win. I had this feeling, the moment that showed up in the news, that AOL’s days were numbered, and I was right. Other companies soon released chat functions, including Google and Facebook, and now AIM is gone. It was a signal that AIM was unsuitable for enterprise messaging, and no one excpet teenagers should bother with it, and wventually the teenagers that did use it, grew out of it, and moved on, and the rest is history.
strict9 8y ago
One of the hardest concepts to explain to non-programmer friends/family is that deleting anything in an app is usually flipping a boolean flag in the database from False to True. And even for the rare cases of actually deleting the record or updating the field value, it likely exists in backups somewhere. It's never gone.
- brootstrap 8y ago
  haha. user.is_deleted = True... A lesson on how to delete user data but still keep it. RE the headline, i'm not sure. "News flash, mega tech corp thats business model is based on accumulating data has been accumulating lots of user data."
- propogandist 8y ago
  That may've been true before, but in a post-GDPR world, you better delete it (assuming you're serving European users) -- unless of-course there's a legal reason to hold onto the data.
- cdoxsey 8y ago
  Twitter actually tries to delete stuff: https://twitter.com/en/privacy https://twitter.com/en/privacy > We keep Log Data for a maximum of 18 months. If you follow the instructions here (or for Periscope here), your account will be deactivated and then deleted. When deactivated, your Twitter account, including your display name, username, and public profile, will no longer be viewable on Twitter.com, Twitter for iOS, and Twitter for Android. For up to 30 days after deactivation it is still possible to restore your Twitter account if it was accidentally or wrongfully deactivated. And they require developers to delete them too: https://developer.twitter.com/en/developer-terms/agreement-and-policy.html https://developer.twitter.com/en/developer-terms/agreement-a... > If Twitter Content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the Twitter Service (including removal of location information), you will make all reasonable efforts to delete or modify such Twitter Content (as applicable) as soon as reasonably possible, and in any case within 24 hours after a request to do so by Twitter or by a Twitter user with regard to their Twitter Content, unless otherwise prohibited by applicable law or regulation, and with the express written permission of Twitter. When I worked at Gnip this requirement was a constant headache for customers, precisely because deleting data is so hard.
deleted 8y ago
[deleted]
eberkund 8y ago
I recently learned to my surprise that Exchange has a feature that allows you to recall emails from the recipients inbox.
- porpoisely 8y ago
  That pretty much only applies to emails sent within the same organization/network. If you send an email outside your organization/network, you generally can't recall those back.
  - Thlom 8y ago
    I guess it can be configured to send a "retraction" e-mail to the recipient? I've received some of those at least.
cantrevealname 8y ago
Skype too. Skype stores your voice mails and video messages forever[1]. I’m not sure about text messages, but I wouldn’t be surprised if they store all messages even if both the sender and recipient deleted them. [1] Details: Clicking on Preferences -> Privacy -> Delete history (OS X), or Options -> Privacy Settings -> Clear history (Windows) pretends to delete the voice/video messages but it merely hides them from your view. If you re-install Skype on the same computer or run Skype on a different computer, all those "deleted" voice mails and video messages re-appear. The delete and clear buttons don’t do what they claim.
Nanocurrency 8y ago
I don't think anyone using these social media networks expects perfect privacy, therefore I have no idea why this article is so popular. We'll have close to perfect privacy in some networks running on Ethereum, 10 years from now. You can quote me on this one.
- kbody 8y ago
  Good luck with having your hopes for anything on Ethereum.

130 points by RobertSmith 8y ago

maxxxxx 8y ago

Isn't it pretty safe to assume that nothing will ever get really deleted? I wonder if the GDPR will change this.

nightfly 8y ago
My first thought when people bring up deleting things is backup tapes. Sure it might or might not be "deleted" from the live data set, but it's almost always gonna live on in backups for probably forever.
- NullPrefix 8y ago
  GDPR affects backup tapes too
  - kradroy 8y ago
    It does, but not like you'd think. If the backup is dormant, the right-to-be-forgotten portions can remain there. Once it comes out of cold storage, those RTBF requests have to be processed over the restored data. Although, in most cases, the dormant backups will expire and be deleted as per a data retention plan.
    - msh 8y ago
      That does not seem to be entirely correct: https://blog.quantum.com/backup-administrators-the-1-advice-to-deal-with-gdpr-and-the-right-of-erasure/#.XGfzF6TnvDu https://blog.quantum.com/backup-administrators-the-1-advice-...
      - guitarbill 8y ago
        Which part isn't correct? What op described sounds like a good and easy way to do it (as long as you have sane retention periods) and doesn't seem to conflict with the article you linked in any way (in fact, they agree).
        msh 8y ago
        In addition you have to tell the customer how long you retain your backups (and only retain them for a reasonable time) and only restore for purely technical reasons.
    - fh973 8y ago
      This is usually solved by encrypting and throwing away the key or seed to effectively delete. Tape is no excuse to violating data retention policies.
- porpoisely 8y ago
  I agree. I find it hard to believe that anyone is going to transport all the physical tapes from cold storage and look through all the physical tapes. My guess is that the data is removed on production and recovery sites. The data will be left on tape since they will most likely never see the light of day again.
  - fh973 8y ago
    This is usually solved by encrypting and throwing away the key or seed to effectively delete.
    - code_duck 8y ago
      Doesn’t that just effectively delete it until it is common to have sufficient computing power to crack the encryption?
brootstrap 8y ago
If people are using a real database, likely there is just a boolean flag for that record getting flipped to True/False to signify deletion. Unless you use mongo, then your shit is probably gone already and you dont even know it.
- wbronitsky 8y ago
  Wait what? Why are you randomly crapping on Mongo? You can use Mongo in exactly the same way as you described.
  - pizza 8y ago
    It's hacker news, randomly crapping on technology nobody brought up is what hackers do (/s)
  - thanatos_dem 8y ago
    Because crapping on Mongo is “in” right now. Years ago it used to be “in” to crap on it for its deaign choices, especially its tendency to lose data (seems to be what the parent comment is referencing; see [1] for a summary of some issues from 2013). More recently it’s “in” for their business decisions, with the relicensing and the rise of API-compatible alternatives pushing some users away. But yeah, really no reason to mock them in this discussion, apart from going for internet cool points from an audience that may also be critical of Mongo. [1] https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb
- talaketu 8y ago
  Real database supports real erasure.

ddebernardy 8y ago

It's a bit disingenuous to expect Twitter or any other social media to actually delete deleted data, except perhaps through a GDPR request or similar. Plus, if your account gets hacked and deleted, or if you delete the account and rejoin, you might be very happy that they didn't actually delete anything and are able to restore your account. (I'm not a fan of social networks or what they do with our data, but the database administrator in me tends to side with them on that front. Keeping data around is a lesser evil than losing it.)

spudlyo 8y ago
Your instincts are correct. I happen to know that DMs at Twitter are (and most likely still) stored in a sharded MySQL store[0]. Deleted DMs and tweets are "tombstoned", and not actually deleted via a DELETE, which has pretty terrible performance characteristics in MySQL [0]: http://highscalability.com/blog/2011/12/19/how-twitter-stores-250-million-tweets-a-day-using-mysql.html http://highscalability.com/blog/2011/12/19/how-twitter-store...
- Operyl 8y ago
  I wonder what they use to generate IDs! /s Man that comment popped up a bunch in that blog post, lol.
wavesounds 8y ago
It's not disingenuous at all. Snapchat deletes all messages, photos and videos by default, either immediately after their received or within 24 hours.
talaketu 8y ago
That's a very wacky take. A general privacy principle is that if the user requests the service provider to delete the user's data, then the service provider should comply, unless there are legal injunctions. It's arguable about user expectation for direct messages on twitter, given there is a sender and receiver and Twitter in the middle, but what is not arguable is that if both users delete the message or their accounts, then the messages should not be retained by the Twitter.

B-Con 8y ago

A lot of the article focuses on DMs sent from eventually deleted/suspend accounts being accessible from the receiving account. When you send a message to another account, it's in their account now. Of course it's not going to disappear along with your account. This had been the canonical behavior of messaging clients for forever. Email obviously, but also IM, chat find, etc. (Trying to remember about BBSs.) Once you press send it's not yours anymore. This should 100% be the expectation of users anywhere they go. Once you press send, do not count on being able to withdraw it under any circumstances. Does this genuinely surprise people or is it a slow news day at TechCrunch?

nikanj 8y ago
The same went for letters too, back when those were still being written.
- zapzupnz 8y ago
  With the difference being that nobody had a copy of your letters, just the recipient.
  - chiefalchemist 8y ago
    Not to sound like an ahole but I presume Twitter would argue it's __their__ message (in the sense that it's on their platform). with the sender and recipient having access to it. Once you hit submit you've forfeited "direct ownership" of the message.
    - mdhshjdjsh 8y ago
      AIM (AOL Instant Messenger) had a similar policy, which, I think, if I remember correctly, was a defining moment for AIM, and once again, for AOL. The EULA for AIM was such that your messages became AOL’s intellectual property. Not only that you could not sue AOL Time Warner for possibly scripting and producing a movie and thus profiting from one of your instant message conversations or chat transcripts, but that maybe they could sue you for trying to do the same, and win. I had this feeling, the moment that showed up in the news, that AOL’s days were numbered, and I was right. Other companies soon released chat functions, including Google and Facebook, and now AIM is gone. It was a signal that AIM was unsuitable for enterprise messaging, and no one excpet teenagers should bother with it, and wventually the teenagers that did use it, grew out of it, and moved on, and the rest is history.

strict9 8y ago

One of the hardest concepts to explain to non-programmer friends/family is that deleting anything in an app is usually flipping a boolean flag in the database from False to True. And even for the rare cases of actually deleting the record or updating the field value, it likely exists in backups somewhere. It's never gone.

brootstrap 8y ago
haha. user.is_deleted = True... A lesson on how to delete user data but still keep it. RE the headline, i'm not sure. "News flash, mega tech corp thats business model is based on accumulating data has been accumulating lots of user data."
propogandist 8y ago
That may've been true before, but in a post-GDPR world, you better delete it (assuming you're serving European users) -- unless of-course there's a legal reason to hold onto the data.
cdoxsey 8y ago
Twitter actually tries to delete stuff: https://twitter.com/en/privacy https://twitter.com/en/privacy > We keep Log Data for a maximum of 18 months. If you follow the instructions here (or for Periscope here), your account will be deactivated and then deleted. When deactivated, your Twitter account, including your display name, username, and public profile, will no longer be viewable on Twitter.com, Twitter for iOS, and Twitter for Android. For up to 30 days after deactivation it is still possible to restore your Twitter account if it was accidentally or wrongfully deactivated. And they require developers to delete them too: https://developer.twitter.com/en/developer-terms/agreement-and-policy.html https://developer.twitter.com/en/developer-terms/agreement-a... > If Twitter Content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the Twitter Service (including removal of location information), you will make all reasonable efforts to delete or modify such Twitter Content (as applicable) as soon as reasonably possible, and in any case within 24 hours after a request to do so by Twitter or by a Twitter user with regard to their Twitter Content, unless otherwise prohibited by applicable law or regulation, and with the express written permission of Twitter. When I worked at Gnip this requirement was a constant headache for customers, precisely because deleting data is so hard.

deleted 8y ago

[deleted]

eberkund 8y ago

I recently learned to my surprise that Exchange has a feature that allows you to recall emails from the recipients inbox.

porpoisely 8y ago
That pretty much only applies to emails sent within the same organization/network. If you send an email outside your organization/network, you generally can't recall those back.
- Thlom 8y ago
  I guess it can be configured to send a "retraction" e-mail to the recipient? I've received some of those at least.

cantrevealname 8y ago

Skype too. Skype stores your voice mails and video messages forever[1]. I’m not sure about text messages, but I wouldn’t be surprised if they store all messages even if both the sender and recipient deleted them. [1] Details: Clicking on Preferences -> Privacy -> Delete history (OS X), or Options -> Privacy Settings -> Clear history (Windows) pretends to delete the voice/video messages but it merely hides them from your view. If you re-install Skype on the same computer or run Skype on a different computer, all those "deleted" voice mails and video messages re-appear. The delete and clear buttons don’t do what they claim.

Nanocurrency 8y ago

I don't think anyone using these social media networks expects perfect privacy, therefore I have no idea why this article is so popular. We'll have close to perfect privacy in some networks running on Ethereum, 10 years from now. You can quote me on this one.

kbody 8y ago
Good luck with having your hopes for anything on Ethereum.