6 ms·

Hetzner deleted all of my vms after getting erroneously flagged by their abuse department due to my account details. (nothing to do with my machines) They sen

Hetzner deleted all of my vms after getting erroneously flagged by their abuse department due to my account details. (nothing to do with my machines) They sent me an email saying my account was paused and to contact them to resolve it, which I did within half an hour. My account was reinstated minus all of my environment.. just a blank account. They deleted everything and told me they had no way to restore it. Beware hosting anything with them unless you keep backups with another provider.

gibsonf1 5y ago
When did this incident happen?
- kjaftaedi 5y ago
  Q1 2018
  - Hetzner_OL 5y ago
    Hi there, I am very sorry that you had this negative experience back in 2018. If memory serves correctly, when we launched Hetzner Cloud during this time, there was a large surge in the number of fake accounts new customers tried to make. So I could imagine our teams being overly careful about new accounts, and especially if names didn't quite match up. Usually, we are able to restore data from locked servers/accounts within the first 48 hours, and to the best of my knowledge, we were able to do that back in 2018 as well. So if you responded within 30 minutes, and your account was re-activated, I think this could have been possible in theory. However, I am not, in fact, a technician or member of the cloud support team, and I am not aware of the details of your case. Perhaps there was something else that happened with your servers or data that I am unaware of. I am very sorry that you had this negative experience with us. Naturally, we don't want any of our customers to feel this way. --Katie, Marketing, Hetzner Online
    - kjaftaedi 5y ago
      Thank you for the response. Your head of sales [name redacted] was the one that reinstated my account and called me "unprofessional" for using a shortened version of my last name. He told me that it is not worth contacting people like me (his words), and suggested there was nothing wrong with their internal procedures and that they had no intention to do anything differently. I understand that mistakes happen, and I do not absolve myself entirely, but I found the response to this particular incident overly offensive and accusatory for a mistake that I do believe to be an error in the way the incident was handled.
      - Hetzner_OL 5y ago
        Hi again, I am sorry that he called you unprofessional. That shouldn't have happened. So I am sorry. His frustration may have been based on a cultural misunderstanding. In Germany, whenever you sign up for a product or a service where there is likely to be an ID check or a validity check, people here know that they need to use their full legal name, even if it is a really long name, or if they have middle name(s) that are part of their legal name. It they are using a credit card, that name needs to match, too. For a German sysadmin, it would be unprofessional to not use the same name as is on their legal ID/payment information. I know that some cultures may abbreviate parts of their middle/last names, so it's understandable that customers from these countries may not be aware of how important it is in Germany for names to match exactly. I know that this issue isn't just an issue in Germany. I used to work in a retail store, and my store was very serious about preventing credit card fraud. We regularly did NOT let people buy things if their name on their ID didn't match the name on their credit card. I also had several Brazilian friends in the USA who had similar issues, and they learned to always use the version of their names on their passports for everything. We will likely still be strict about customers' information matching their IDs/payment information. He's right that we will likely not change this any time soon. However, I am sorry that his answer came off as abrasive and impolite. -Katie
      - dang 5y ago
        (I've redacted the name in the parent - we sometimes do that, especially when there's a risk of doxxing or other sort of personal consequences.)
1cvmask 5y ago
Any details on what the alleged abuse was?
- kjaftaedi 5y ago
  I used a shortened version of my last name for my account details. It's difficult to pronounce, so I usually go by my preferred name. (They have my full name from my e-mail and credit card) I'm most upset that there was no opportunity to resolve this minor confusion amicably, and what really got me the most is that there was no attempt at an apology, considering the e-mail I got clearly stated that service would resume "once steps are taken to remedy the situation" I was very polite to them, but everyone I talked to at the company was telling me that it was my fault. I've never left an encounter with a company feeling more frustrated.
  - thatwasunusual 5y ago
    > I used a shortened version of my last name for my account details. So not Hetzner's fault, then.
  - zyx321 5y ago
    It was your fault. IANAL, but I'm reasonably sure that under German law a contract that you deliberately enter under a wrong name is retroactively void (von Anfang an nichtig). Sorry. When it comes to names, Germany doesn't f'k around.
    - redwood 5y ago
      Well then Americans ought to use their nee Virginia region with caution?
    - kjaftaedi 5y ago
      This may very well be the case, which would only strengthen the need for posting such a warning so that other people are aware that they can have their data wiped due to minor trivialities.
echelon 5y ago
Did you write a blog post or have any screenshots of emails to them? Anecdotes like these are enough to tarnish a brand forever in the minds of potential customers.
- kjaftaedi 5y ago
  Here is basically the core of the communications. https://imgur.com/a/y1dMhd2 https://imgur.com/a/y1dMhd2
  - fooker 5y ago
    Wow this is very unprofessional. I'd think twice before hosting anything with them.
  - mixmastamyk 5y ago
    Yeah, that guy definitely should be fired for addressing a customer that way. Assuming this is all accurate.
  - mfru 5y ago
    If accurate this is a disgusting way of communication on behalf of the Hetzner employee. I would immediately consider to move my projects to another provider if I were a customer with them. They have an account on HN, so maybe they could clarify. I certainly wouldn't be considering to become a client with them.
  - Hetzner_OL 5y ago
    Hi again, I see. Before I respond, could I ask you to please re-post that image without my colleague's name in it? And could you also edit your above comment so that my colleague's name is no longer in it. Unfortunately, some of our coworkers have been harassed in the past after their names were published online. So we try to prevent this from happening. Thanks in advance for doing that. I really appreciate it. Okay my response - Again, I think there is a bit of a cultural misunderstanding going on here. I would say most Germans understand that they are responsible for verifying their identity and using their legal name when making important purchases or doing something else official. Here, names that do not match are often a sign that the account is fake. We find hundreds of fake accounts every week. Fake accounts are very closely linked with criminal behavior, people who otherwise abuse our products, and people who do not follow our ToS. Clearly, we do not want to attract customers with fake accounts to our company. So if we find an account that seems fake, it does not make sense for us to expend extra time and energy to write to that customer and ask them if they made a mistake when they entered their name on their customer account. That would dramatically increase the likelihood that spammers and scammers would get accounts and abuse our services. It would not only hurt our customers (because it would hurt our the trustworthiness of our IPs, etc.), but it would also lead to a massive increase in abuse and to victims affected by that abuse. So, I agree that the tone from my colleague could be better here. And I will talk to him about that. And, again, for that, we are sorry. But the basic content of what he wrote is sound in my opinion. --Katie
janmo 5y ago
That's definitively an unacceptable behaviour.
Bellamy 5y ago
As a long time Hetzner user I find this hard to believe. I have also gotten email from the security department but they respond always really quickly.
- kjaftaedi 5y ago
  I got the feeling that they made a mistake following their own procedures, but I was told many times that this is how they treat everyone. I posted screenshots of some communications in another reply.
19h 5y ago
I’m running $2500 worth of servers on Hetzner privately since 2015 and about $400 worth of VMs on Hetzner Cloud. The only instance of issues with them was Hetzner locking the outgoing traffic of a VM after it started bursting a few million packets to the local network in an infinite loop. I called them, they explained it, I fixed it and they unblocked it. Pretty happy with Hetzner. The traffic that goes over some of my servers would be $50k+ per month on AWS.
- sgt 5y ago
  $50k+ - how do you get to that? That's pretty insane. Does that include the equivalent EC2/etc on AWS?
  - 19h 5y ago
    Current pricing tiering in AWS, as per us-east-1 and eu-central-1, for a total of 950TB: GB * cost per GB 1 * $0 9999 * $0,09 40000 * $0,085 100000 * $0,07 800000 * $0,05 = $51.299,91
  - mnadkvlb 5y ago
    a video by Wendell from level1techs explains the same egress costs on AWS: https://www.youtube.com/watch?v=XRZCIb_9arQ https://www.youtube.com/watch?v=XRZCIb_9arQ
  - mixmastamyk 5y ago
    AWS is known for being expensive. Known for being very expensive for "egress." Sibiling has done the math.
- algo_trader 5y ago
  Just to re-iterate, you are a "regular customer" running 1PB/month outgoing-traffic, at $3000/month and have had no issues?
  - 19h 5y ago
    Yes.
syshum 5y ago
>>Beware hosting anything with them unless you keep backups with another provider. That should be true of any provider / data.. 3 is 1, 2 is none, and 1 is crazy... If you are not following 3-2-1 backup you are asking for trouble I dont care what provider or service you are using
- kjaftaedi 5y ago
  They shut down my account within two weeks of me signing up, so I didn't get a chance to establish a fully production environment. I did however lose a full two weeks of heavy effort.

gibsonf1 5y ago

When did this incident happen?

kjaftaedi 5y ago
Q1 2018
- Hetzner_OL 5y ago
  Hi there, I am very sorry that you had this negative experience back in 2018. If memory serves correctly, when we launched Hetzner Cloud during this time, there was a large surge in the number of fake accounts new customers tried to make. So I could imagine our teams being overly careful about new accounts, and especially if names didn't quite match up. Usually, we are able to restore data from locked servers/accounts within the first 48 hours, and to the best of my knowledge, we were able to do that back in 2018 as well. So if you responded within 30 minutes, and your account was re-activated, I think this could have been possible in theory. However, I am not, in fact, a technician or member of the cloud support team, and I am not aware of the details of your case. Perhaps there was something else that happened with your servers or data that I am unaware of. I am very sorry that you had this negative experience with us. Naturally, we don't want any of our customers to feel this way. --Katie, Marketing, Hetzner Online
  - kjaftaedi 5y ago
    Thank you for the response. Your head of sales [name redacted] was the one that reinstated my account and called me "unprofessional" for using a shortened version of my last name. He told me that it is not worth contacting people like me (his words), and suggested there was nothing wrong with their internal procedures and that they had no intention to do anything differently. I understand that mistakes happen, and I do not absolve myself entirely, but I found the response to this particular incident overly offensive and accusatory for a mistake that I do believe to be an error in the way the incident was handled.
    - Hetzner_OL 5y ago
      Hi again, I am sorry that he called you unprofessional. That shouldn't have happened. So I am sorry. His frustration may have been based on a cultural misunderstanding. In Germany, whenever you sign up for a product or a service where there is likely to be an ID check or a validity check, people here know that they need to use their full legal name, even if it is a really long name, or if they have middle name(s) that are part of their legal name. It they are using a credit card, that name needs to match, too. For a German sysadmin, it would be unprofessional to not use the same name as is on their legal ID/payment information. I know that some cultures may abbreviate parts of their middle/last names, so it's understandable that customers from these countries may not be aware of how important it is in Germany for names to match exactly. I know that this issue isn't just an issue in Germany. I used to work in a retail store, and my store was very serious about preventing credit card fraud. We regularly did NOT let people buy things if their name on their ID didn't match the name on their credit card. I also had several Brazilian friends in the USA who had similar issues, and they learned to always use the version of their names on their passports for everything. We will likely still be strict about customers' information matching their IDs/payment information. He's right that we will likely not change this any time soon. However, I am sorry that his answer came off as abrasive and impolite. -Katie
    - dang 5y ago
      (I've redacted the name in the parent - we sometimes do that, especially when there's a risk of doxxing or other sort of personal consequences.)

1cvmask 5y ago

Any details on what the alleged abuse was?

kjaftaedi 5y ago
I used a shortened version of my last name for my account details. It's difficult to pronounce, so I usually go by my preferred name. (They have my full name from my e-mail and credit card) I'm most upset that there was no opportunity to resolve this minor confusion amicably, and what really got me the most is that there was no attempt at an apology, considering the e-mail I got clearly stated that service would resume "once steps are taken to remedy the situation" I was very polite to them, but everyone I talked to at the company was telling me that it was my fault. I've never left an encounter with a company feeling more frustrated.
- thatwasunusual 5y ago
  > I used a shortened version of my last name for my account details. So not Hetzner's fault, then.
- zyx321 5y ago
  It was your fault. IANAL, but I'm reasonably sure that under German law a contract that you deliberately enter under a wrong name is retroactively void (von Anfang an nichtig). Sorry. When it comes to names, Germany doesn't f'k around.
  - redwood 5y ago
    Well then Americans ought to use their nee Virginia region with caution?
  - kjaftaedi 5y ago
    This may very well be the case, which would only strengthen the need for posting such a warning so that other people are aware that they can have their data wiped due to minor trivialities.

echelon 5y ago

Did you write a blog post or have any screenshots of emails to them? Anecdotes like these are enough to tarnish a brand forever in the minds of potential customers.

kjaftaedi 5y ago
Here is basically the core of the communications. https://imgur.com/a/y1dMhd2 https://imgur.com/a/y1dMhd2
- fooker 5y ago
  Wow this is very unprofessional. I'd think twice before hosting anything with them.
- mixmastamyk 5y ago
  Yeah, that guy definitely should be fired for addressing a customer that way. Assuming this is all accurate.
- mfru 5y ago
  If accurate this is a disgusting way of communication on behalf of the Hetzner employee. I would immediately consider to move my projects to another provider if I were a customer with them. They have an account on HN, so maybe they could clarify. I certainly wouldn't be considering to become a client with them.
- Hetzner_OL 5y ago
  Hi again, I see. Before I respond, could I ask you to please re-post that image without my colleague's name in it? And could you also edit your above comment so that my colleague's name is no longer in it. Unfortunately, some of our coworkers have been harassed in the past after their names were published online. So we try to prevent this from happening. Thanks in advance for doing that. I really appreciate it. Okay my response - Again, I think there is a bit of a cultural misunderstanding going on here. I would say most Germans understand that they are responsible for verifying their identity and using their legal name when making important purchases or doing something else official. Here, names that do not match are often a sign that the account is fake. We find hundreds of fake accounts every week. Fake accounts are very closely linked with criminal behavior, people who otherwise abuse our products, and people who do not follow our ToS. Clearly, we do not want to attract customers with fake accounts to our company. So if we find an account that seems fake, it does not make sense for us to expend extra time and energy to write to that customer and ask them if they made a mistake when they entered their name on their customer account. That would dramatically increase the likelihood that spammers and scammers would get accounts and abuse our services. It would not only hurt our customers (because it would hurt our the trustworthiness of our IPs, etc.), but it would also lead to a massive increase in abuse and to victims affected by that abuse. So, I agree that the tone from my colleague could be better here. And I will talk to him about that. And, again, for that, we are sorry. But the basic content of what he wrote is sound in my opinion. --Katie

janmo 5y ago

That's definitively an unacceptable behaviour.

Bellamy 5y ago

As a long time Hetzner user I find this hard to believe. I have also gotten email from the security department but they respond always really quickly.

kjaftaedi 5y ago
I got the feeling that they made a mistake following their own procedures, but I was told many times that this is how they treat everyone. I posted screenshots of some communications in another reply.

19h 5y ago

I’m running $2500 worth of servers on Hetzner privately since 2015 and about $400 worth of VMs on Hetzner Cloud. The only instance of issues with them was Hetzner locking the outgoing traffic of a VM after it started bursting a few million packets to the local network in an infinite loop. I called them, they explained it, I fixed it and they unblocked it. Pretty happy with Hetzner. The traffic that goes over some of my servers would be $50k+ per month on AWS.

sgt 5y ago
$50k+ - how do you get to that? That's pretty insane. Does that include the equivalent EC2/etc on AWS?
- 19h 5y ago
  Current pricing tiering in AWS, as per us-east-1 and eu-central-1, for a total of 950TB: GB * cost per GB 1 * $0 9999 * $0,09 40000 * $0,085 100000 * $0,07 800000 * $0,05 = $51.299,91
- mnadkvlb 5y ago
  a video by Wendell from level1techs explains the same egress costs on AWS: https://www.youtube.com/watch?v=XRZCIb_9arQ https://www.youtube.com/watch?v=XRZCIb_9arQ
- mixmastamyk 5y ago
  AWS is known for being expensive. Known for being very expensive for "egress." Sibiling has done the math.
algo_trader 5y ago
Just to re-iterate, you are a "regular customer" running 1PB/month outgoing-traffic, at $3000/month and have had no issues?
- 19h 5y ago
  Yes.

syshum 5y ago

>>Beware hosting anything with them unless you keep backups with another provider. That should be true of any provider / data.. 3 is 1, 2 is none, and 1 is crazy... If you are not following 3-2-1 backup you are asking for trouble I dont care what provider or service you are using

kjaftaedi 5y ago
They shut down my account within two weeks of me signing up, so I didn't get a chance to establish a fully production environment. I did however lose a full two weeks of heavy effort.