7 ms·

Herokai here. Unfortunately we had no choice on the data retention front — once we’ve disconnected your database, we aren’t ALLOWED to hold your data for more t

Herokai here. Unfortunately we had no choice on the data retention front — once we’ve disconnected your database, we aren’t ALLOWED to hold your data for more than 30 days. That’s part of the data scrubbing protocol that we agree to when you sign up. We fought hard for 90+ days internally, but in the end couldn’t get over the issue that we’d be in violation of our contracts with customers.

craigkerstiens 4y ago
Having worked at Heroku and had a large part in building Heroku Postgres I do not recall this explicit policy, and it seems very squirrelly to me. Maybe this came in as a policy in recent years and it is the case, but still seems like hiding behind a policy as opposed to doing right by customers. You could easily block all incoming connections to the database. For a free database of 10k rows there were no SLAs, and you would still technically be hosting the database. Even taking a dump and emailing it to me feels like a safer option here. There were better answers here for sure. If the honest answer is we just didn't feel the effort was worth it for this class of users at least own that.
- robryan 4y ago
  Yeah ideally hold onto a backup for say a year, if the owner hasn't come and downloaded it after a year can then assume that they don't want it.
  - johannes1234321 4y ago
    It's not that easy. Those databases might contain personal information which is protected under different privacy laws like GDPR, HIPAA and others and Heroku/Salesforce can not simply store that for longer than agreed upon and Heroku cancelling the account enables the retention period as the customer agreed upon as part of the T&C.
    - tuukkah 4y ago
      The account was not cancelled.
    - jbverschoor 4y ago
      I wonder if you guys delete email addresses from people who ask to unsubscribe , or when I tell t you to delete my information, according to gdpr Probably not, because it’s ” difficult “
      - johannes1234321 4y ago
        Many avoid it, but once they have to deal with a (ex-)customer's lawyer they learn what is even more difficult.
- tptacek 4y ago
  Having been through a SOC2 audit: this wouldn't fly. It's on the checklist of issues that you get hit with regardless of what kind of company you are: when customer accounts are terminated, the data retention clock starts ticking. You can pick an arbitrary time frame for retention, but whatever you pick, you have to communicate to users, and you can't just change it on a whim. Normal customers want this clock short. They don't want you to retain their stuff after they cancel.
  - shkkmo 4y ago
    The customer account wasn't terminated, the free DB being used by paid Dynos was deleted without any input from or notification to the customer. I highly doubt normal customers want this clock short when the cancellation is not customer initiated.
    - tptacek 4y ago
      Data retention policies are written by compliance people, not product people, so distinctions like overt, deliberate cancellation and "cancellation for nonpayment" or "abandonment" or "discontinuation" usually aren't captured in them. I'm not saying it's great that Heroku deleted these databases; I'm saying: the description given upthread, that the databases were deleted because of contractual requirements, is super plausible.
      - shkkmo 4y ago
        Plausible, sure, but that doesn't mean it isn't a cop out. There are many things that Heroku could have done to prevent this, like delaying the disconnection when they don't have confirmed delivery of the notifications and/or when they are connected to paid dynos/accounts. Thus didn't happen because of the contract, but because the people implementing this transition didn't give a crap.
        tptacek 4y ago
        I'm just telling you that what the commenter upthread said rang true. I'm not offering an assessment of compliance regimes.
- CoastalCoder 4y ago
  > Even taking a dump and emailing it to me feels like a safer option here. I genuinely had to read this twice to get the intended meaning.
  - yuubi 4y ago
    as someone who doesn't use heroku, so disregard my opinion: i'd probably prefer feces-by-email to surprise database deletion
- deleted 4y ago
  [deleted]
- FPGAhacker 4y ago
  There is a legal difference between a company policy and a contract with a customer.
mytailorisrich 4y ago
Please. You'd just need to ask if the customer is OK with 90 days instead of 30. Done. The company has no commercial interest in doing that, though.
- porpoisemonkey 4y ago
  It doesn't sound like this would have helped in this particular case since they were unable to contact the customer.
  - mytailorisrich 4y ago
    I was replying to a bullshit claim that they cannot retain for more than 30 days no matter what. That's hiding behind the T&Cs instead of owning their decision not to even try because there is nothing in it for them.
    - joemi 4y ago
      So you're in favor of companies breaking their terms and conditions at will? I think that would cause quite a lot more outrage and problems.
      - mytailorisrich 4y ago
        You did not read my comments, did you? The T&Cs are an agreement between the parties. That agreement can be changed at any time if both parties agree. So they just need to ask.
        joemi 4y ago
        But if they make an exception for one person, they're opening the doors to complaints and possibly even legal action from others who want the same change. So making an exception for one person isn't really making an exception for just one person. Instead, it's a large process that needs careful legal consideration.
      - 4y ago
fireworks 4y ago
Are you allowed to inform paying customers that you are going to do this? This is my primary complaint here. I don't understand how this oversight happened. This is going to cause an enormous amount of time and energy to recover from this.
- yamtaddle 4y ago
  > Are you allowed to inform paying customers that you are going to do this? I can't be the only one who's basically completely blind to emails from major companies, including SaaS providers, because they're so fucking spammy that the SNR is like 1:99. Notifying me by email, for one of these places, is functionally the same as not notifying me at all. [EDIT] Sorry, didn't mean to imply the parent wasn't paying attention, just that I'd fully expect a very high percentage of their users to miss the warning in all the noise even if they emailed everyone—even if they emailed them a couple times, actually. That's the cost of every company sending out tons of "join our online seminar on [product]!" and "hey, look, it's our newsletter you never read!" and "it's time for our weekly TOS modification!" emails.
  - fireworks 4y ago
    I'm not blind, they didn't send one. And they admitted to me that they did not send one.
    - yamtaddle 4y ago
      Well, that's even worse, of course.
    - eptcyka 4y ago
      That's a major failure on their part. I did get several nagging emails from them.
    - ehPReth 4y ago
      Even if they did send it, email isn't guaranteed. What if it went to spam or was otherwise mishandled or missed?
numpad0 4y ago
Using the fact that a customer was shown exit as the basis for destroying their assets don’t look great to me, at least on surface…
dboreham 4y ago
That's peak idiocy and the product of lawyers taking over the assylum.
jacobsenscott 4y ago
If you enable daily backup those are nuked too?
- csomar 4y ago
  That would be interesting to learn. Backups will be just an additional surcharge rather than a "real backup strategy".
rurp 4y ago
If I'm understanding you correctly, the 30 day policy is one that Heroku chose to put in the contract. Engineering might have fought the terms, and yes they need to be followed once set, but it seems totally fair to blame Heroku for creating the limitation in the first place.
- mst 4y ago
  When they were written, short sighted acquirers yeeting the free tier was likely not something the people writing the relevant clauses were even considering as a possibility, and honestly it's such a ridiculous decision from a commercial perspective that I find it hard to assign blame for not foreseeing it. Plus, it would all likely have worked out fine if they'd emailed the customer a warning or three like they intended to do - it was the failure to do so combined with the failure to detect and remediate the initial failure that sent things down such a dark path here.
inopinatus 4y ago
This sounds like the legal equivalent of looking the wrong way through a telescope. Whoever fostered that naive interpretation was a nitwit. If they’re an actual lawyer, they promoted an intentional, mutually harmful unilateral reinterpretation of an agreement and should be sacked. Cowering behind T&Cs like this is intellectual bankruptcy. There’s always another solution. The law is not a programming language.
tacker2000 4y ago
[flagged]
nimchimpsky 4y ago
[dead]
skissane 4y ago
> We fought hard for 90+ days internally, but in the end couldn’t get over the issue that we’d be in violation of our contracts with customers. Contracts with some customers, surely? You could have the default be 90+ days, then those customers whose contracts specify a shorter timeframe get that shorter timeframe configured on their account instead. You could give the customer the choice at signup, and let them change it later using the settings console. If their contract doesn't specify a period, send them a notification that you will be changing it to 90+ days, but telling them they have the right to object if they disagree with that.
ehPReth 4y ago
Every single executive in charge of this decision of how to handle precious customer data at Heroku feel be completely ashamed and take a long, hard look in the mirror.
dotancohen 4y ago
The phrase "aren't allowed" supposes some regulatory agency forbidding an action. When it's your own internal policy that contradicts the action, the proper term is "won't".
Cheezewheel 4y ago
No choice? That's just the way it is, it can't be helped? Did God himself come down and decree that Herokai *MUST* only hold data for 30 days? Did the FBI come in an threadten to charge your executives with sedition? Yea, no. You decided to make the decision for contracts to be that way. The fact that you "fought hard" but that decided on the 30 day retention anyways means that clearly the opinions of engineers don't matter and that the company is completely captured by the lawyers and out of touch executives. It hardly inspires confidence. It also doesn't at all address the fact that you failed to contact an apparently paying customer that their data was about to be nuked, contract or no.

craigkerstiens 4y ago

Having worked at Heroku and had a large part in building Heroku Postgres I do not recall this explicit policy, and it seems very squirrelly to me. Maybe this came in as a policy in recent years and it is the case, but still seems like hiding behind a policy as opposed to doing right by customers. You could easily block all incoming connections to the database. For a free database of 10k rows there were no SLAs, and you would still technically be hosting the database. Even taking a dump and emailing it to me feels like a safer option here. There were better answers here for sure. If the honest answer is we just didn't feel the effort was worth it for this class of users at least own that.

robryan 4y ago
Yeah ideally hold onto a backup for say a year, if the owner hasn't come and downloaded it after a year can then assume that they don't want it.
- johannes1234321 4y ago
  It's not that easy. Those databases might contain personal information which is protected under different privacy laws like GDPR, HIPAA and others and Heroku/Salesforce can not simply store that for longer than agreed upon and Heroku cancelling the account enables the retention period as the customer agreed upon as part of the T&C.
  - tuukkah 4y ago
    The account was not cancelled.
  - jbverschoor 4y ago
    I wonder if you guys delete email addresses from people who ask to unsubscribe , or when I tell t you to delete my information, according to gdpr Probably not, because it’s ” difficult “
    - johannes1234321 4y ago
      Many avoid it, but once they have to deal with a (ex-)customer's lawyer they learn what is even more difficult.
tptacek 4y ago
Having been through a SOC2 audit: this wouldn't fly. It's on the checklist of issues that you get hit with regardless of what kind of company you are: when customer accounts are terminated, the data retention clock starts ticking. You can pick an arbitrary time frame for retention, but whatever you pick, you have to communicate to users, and you can't just change it on a whim. Normal customers want this clock short. They don't want you to retain their stuff after they cancel.
- shkkmo 4y ago
  The customer account wasn't terminated, the free DB being used by paid Dynos was deleted without any input from or notification to the customer. I highly doubt normal customers want this clock short when the cancellation is not customer initiated.
  - tptacek 4y ago
    Data retention policies are written by compliance people, not product people, so distinctions like overt, deliberate cancellation and "cancellation for nonpayment" or "abandonment" or "discontinuation" usually aren't captured in them. I'm not saying it's great that Heroku deleted these databases; I'm saying: the description given upthread, that the databases were deleted because of contractual requirements, is super plausible.
    - shkkmo 4y ago
      Plausible, sure, but that doesn't mean it isn't a cop out. There are many things that Heroku could have done to prevent this, like delaying the disconnection when they don't have confirmed delivery of the notifications and/or when they are connected to paid dynos/accounts. Thus didn't happen because of the contract, but because the people implementing this transition didn't give a crap.
      - tptacek 4y ago
        I'm just telling you that what the commenter upthread said rang true. I'm not offering an assessment of compliance regimes.
CoastalCoder 4y ago
> Even taking a dump and emailing it to me feels like a safer option here. I genuinely had to read this twice to get the intended meaning.
- yuubi 4y ago
  as someone who doesn't use heroku, so disregard my opinion: i'd probably prefer feces-by-email to surprise database deletion
deleted 4y ago
[deleted]
FPGAhacker 4y ago
There is a legal difference between a company policy and a contract with a customer.

mytailorisrich 4y ago

Please. You'd just need to ask if the customer is OK with 90 days instead of 30. Done. The company has no commercial interest in doing that, though.

porpoisemonkey 4y ago
It doesn't sound like this would have helped in this particular case since they were unable to contact the customer.
- mytailorisrich 4y ago
  I was replying to a bullshit claim that they cannot retain for more than 30 days no matter what. That's hiding behind the T&Cs instead of owning their decision not to even try because there is nothing in it for them.
  - joemi 4y ago
    So you're in favor of companies breaking their terms and conditions at will? I think that would cause quite a lot more outrage and problems.
    - mytailorisrich 4y ago
      You did not read my comments, did you? The T&Cs are an agreement between the parties. That agreement can be changed at any time if both parties agree. So they just need to ask.
      - joemi 4y ago
        But if they make an exception for one person, they're opening the doors to complaints and possibly even legal action from others who want the same change. So making an exception for one person isn't really making an exception for just one person. Instead, it's a large process that needs careful legal consideration.
    - 4y ago

fireworks 4y ago

Are you allowed to inform paying customers that you are going to do this? This is my primary complaint here. I don't understand how this oversight happened. This is going to cause an enormous amount of time and energy to recover from this.

yamtaddle 4y ago
> Are you allowed to inform paying customers that you are going to do this? I can't be the only one who's basically completely blind to emails from major companies, including SaaS providers, because they're so fucking spammy that the SNR is like 1:99. Notifying me by email, for one of these places, is functionally the same as not notifying me at all. [EDIT] Sorry, didn't mean to imply the parent wasn't paying attention, just that I'd fully expect a very high percentage of their users to miss the warning in all the noise even if they emailed everyone—even if they emailed them a couple times, actually. That's the cost of every company sending out tons of "join our online seminar on [product]!" and "hey, look, it's our newsletter you never read!" and "it's time for our weekly TOS modification!" emails.
- fireworks 4y ago
  I'm not blind, they didn't send one. And they admitted to me that they did not send one.
  - yamtaddle 4y ago
    Well, that's even worse, of course.
  - eptcyka 4y ago
    That's a major failure on their part. I did get several nagging emails from them.
  - ehPReth 4y ago
    Even if they did send it, email isn't guaranteed. What if it went to spam or was otherwise mishandled or missed?

numpad0 4y ago

Using the fact that a customer was shown exit as the basis for destroying their assets don’t look great to me, at least on surface…

dboreham 4y ago

That's peak idiocy and the product of lawyers taking over the assylum.

jacobsenscott 4y ago

If you enable daily backup those are nuked too?

csomar 4y ago
That would be interesting to learn. Backups will be just an additional surcharge rather than a "real backup strategy".

rurp 4y ago

If I'm understanding you correctly, the 30 day policy is one that Heroku chose to put in the contract. Engineering might have fought the terms, and yes they need to be followed once set, but it seems totally fair to blame Heroku for creating the limitation in the first place.

mst 4y ago
When they were written, short sighted acquirers yeeting the free tier was likely not something the people writing the relevant clauses were even considering as a possibility, and honestly it's such a ridiculous decision from a commercial perspective that I find it hard to assign blame for not foreseeing it. Plus, it would all likely have worked out fine if they'd emailed the customer a warning or three like they intended to do - it was the failure to do so combined with the failure to detect and remediate the initial failure that sent things down such a dark path here.

inopinatus 4y ago

This sounds like the legal equivalent of looking the wrong way through a telescope. Whoever fostered that naive interpretation was a nitwit. If they’re an actual lawyer, they promoted an intentional, mutually harmful unilateral reinterpretation of an agreement and should be sacked. Cowering behind T&Cs like this is intellectual bankruptcy. There’s always another solution. The law is not a programming language.

tacker2000 4y ago

[flagged]

nimchimpsky 4y ago

[dead]

skissane 4y ago

> We fought hard for 90+ days internally, but in the end couldn’t get over the issue that we’d be in violation of our contracts with customers. Contracts with some customers, surely? You could have the default be 90+ days, then those customers whose contracts specify a shorter timeframe get that shorter timeframe configured on their account instead. You could give the customer the choice at signup, and let them change it later using the settings console. If their contract doesn't specify a period, send them a notification that you will be changing it to 90+ days, but telling them they have the right to object if they disagree with that.

ehPReth 4y ago

Every single executive in charge of this decision of how to handle precious customer data at Heroku feel be completely ashamed and take a long, hard look in the mirror.

dotancohen 4y ago

The phrase "aren't allowed" supposes some regulatory agency forbidding an action. When it's your own internal policy that contradicts the action, the proper term is "won't".

Cheezewheel 4y ago

No choice? That's just the way it is, it can't be helped? Did God himself come down and decree that Herokai *MUST* only hold data for 30 days? Did the FBI come in an threadten to charge your executives with sedition? Yea, no. You decided to make the decision for contracts to be that way. The fact that you "fought hard" but that decided on the 30 day retention anyways means that clearly the opinions of engineers don't matter and that the company is completely captured by the lawyers and out of touch executives. It hardly inspires confidence. It also doesn't at all address the fact that you failed to contact an apparently paying customer that their data was about to be nuked, contract or no.