7 ms·

Efnet klines all Hetzner netblocks

97 points by alternize 14y ago

mschuster91 14y ago
Uhhh... that's bad. Real bad, for both sides. I can totally understand Hetzner for just forwarding abuse complaints to the client (for root servers, the hoster usually has no "emergency ssh keys"), so the faster the original owner of the server can boot out the hacker, the better. At least it's better than disconnecting the customer from the internet entirely, especially as "haxx0ring" a server is damn easy these days, given the numbers of aged Wordpress installs alone. Also, a server owner who knows his server can support me as a hacking victim better than the hoster support who often knows nothing about configuration details, OS, disk encryption etc. on the server. But I also understand EFNet, that their emails got blindly forwarded is bad, too...
- OGC 14y ago
  AFAIK, aged wordpress install usually run on shared hosting.
  - mschuster91 14y ago
    There are quite a number of webdev/media agencies which run the Wordpress sites of their customers on their own servers, but neglect the maintenance.
- claudius 14y ago
  > I can totally understand Hetzner for just forwarding abuse complaints to the client (for root servers, the hoster usually has no "emergency ssh keys"), so the faster the original owner of the server can boot out the hacker, the better. Removing confidential information from such an email should take ~2min, I think that’s time well-spent, especially given that email per se is not a real-time medium.
  - mschuster91 14y ago
    I guess their automated forward system regex-greps the IP out of the complaint email and then forwards the mail to the applicable customer(s), so this system actually is realtime. Also, it places the burden of dealing with spam on the customer, so that the hoster doesn't have to do spam-filtering.
codesuela 14y ago
> Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception. They have realized they can save money (by limiting attacks) by redirecting the attacks back at the person reporting them. That way, the hacker/cracker/kiddie using their services will not cancel their contract with Hetzner, and in return Hetzner will remain protected. I don't think that this has to do with Hetzner needing criminal business but rather with Hetzner not wanting to shut down an entire server if a part of it has been breached. Forwarding that complaint in it's entirety is definitely not best practice however making such allegations is neither.
- nmcfarl 14y ago
  That was roughly my take away. Hetzner looks really bad - but the explanation that they are incompetent and lazy sounds more likely than the criminal explanation. And EFnet looks wronged. But also looks bad for spreading allegations against Hetzner of profiteering from criminals without much more substantial backing. Sticking to the facts of the situation would have produced a more powerful, and damning condemnation.
  - rhizome 14y ago
    This isn't court, I think it's reasonable to forgive humanity entering into the issue. Besides, this appears to be a citizenship issue, so banishment also comes as no surprise.
- devicenull 14y ago
  What are "best practices" here? I've been unable to find any guidance about how these types of issues should be handled.
subsystem 14y ago
"This has worked very well due to our personal involvement with a lot of said organizations. If we find an abuser on IRC, we try to not only ban him or her, but also to contact the provider so that the problem is handled at the right end, often with the involvement of law enforcement, as was the case with Kevin Mitnick, t0rn and a lot of other well publicized hackers/crackers." I'm not sure that is something to be proud of.
- ihsw 14y ago
  It is a necessary sacrifice to foster an open and mutually beneficial service.
- lawnchair_larry 14y ago
  Yeah, I don't recall Mitnick participating in extortion either. This author is a piece of work.
- nknighthb 14y ago
  Misconduct by authorities does not change the clear criminal culpability of Mitnick.
- laumars 14y ago
  Of course it is. If someone cost you time and money and severely harassed users of a service you're offering by illegally attacking your servers, then I bet you'd be all for reporting them as well. The problem with script kiddies is that they're too lazy to create anything of their own and thus cannot emphasis with how hard it is to set up and maintain a free public service. So they don't have any second thought about destroying other peoples hard work. Plus the pseudo-anonymity of being online and the ability to not look your victims in the eye make such crimes easy for even some of the more ethical kids to shrug off. The only way to have an open internet where everyone is free to create and contribute is to rat out those who seek to destroy it. People like the aforementioned might enjoy all the perks of an open internet, but their actions have the opposite effect. So I'm not only in favour of reporting abusers, but have actually reported attacks on my own servers to their respective ISPs.
ciupicri 14y ago
The article doesn't seem to mention if the sensitive information from the email was marked as such.
- TazeTSchnitzel 14y ago
  Why would it need to be? Surely Hetzner and the like would be familiar with how sensitive it could be.
  - DangerousPie 14y ago
    If you consider nicknames sensitive you are gonna have to mention that, because a lot of people wouldn't.
    - TazeTSchnitzel 14y ago
      Hetzner, as a hosting provider, would likely be well aware of the workings of IRC networks, so I doubt that.
eliasmacpherson 14y ago
Doesn't explain why efnet themselves are not at fault for failing to redact the details if they are so concerned. What if the attacker was Hetzner staff?
- ahnberg 14y ago
  Should we perhaps have sent the request through anonymized pidgeon? When you send an e-mail to someone you can find out quite a lot of information from headers, from a nick or a given name. When contacting ISPs some people attach contact information so that the ISP can request more information, maybe even make a call to verify identity and discuss the matter to help resolve it. This doesn't mean that the information is intended for the abuser himself. I'd expect the ISP to send anonymized information/questions to the owner of said service, and not just forward "everything" (especially without discussion with the reporting party). Reporting DDoS and abusers risks putting a huge target on yourself, your company, your servers, etc. That is what the issue is about.
  - eliasmacpherson 14y ago
    I specified redacting the details, i.e. the irc nickname. If the information within the headers is that sensitive, the headers should probably not be forwarded from efnet to hetzner. At its most basic their servers are attacking efnets and so should be considered hostile. By all means pass on contact information to resolve amicably - but don't leave sensitive information. There's no reason to pass on irc nicks or sensitive email addresses - what use are they to hetzner? I have the same expectation of efnet as you have for the ISP. If the issue is that efnet is being identified as the reporter to the rogue attacker, then that is not what is stated in the efnet admin's text.
    - ahnberg 14y ago
      EFnet didn't pass anything on. EFnet reported abuse, and the EFnet reporting staff members information was the information that was leaked/forwarded to the abuser. Not a third party to whom EFnet acted proxy. You seem to misunderstand the situation.
itry 14y ago
Can somebody tell me, what "kline" means? I never heard that word and dont find a dictionary entry.
- dpcx 14y ago
  http://en.wikipedia.org/wiki/K-line_(IRC)#K-line http://en.wikipedia.org/wiki/K-line_(IRC)#K-line
- TazeTSchnitzel 14y ago
  kline is short for "kill line" and refers to lines in configuration files for IRC daemons (IRCd), i.e. IP ban.
- akx 14y ago
  kline (K-line) basically [K]ills a network or IP block, forbidding them from connecting to the IRC server. (I think the term stems from the config files for the original ircd, circa 1988)
- kenny_r 14y ago
  It's IRC terminology: http://en.wikipedia.org/wiki/IRCd#K-line http://en.wikipedia.org/wiki/IRCd#K-line
- deleted 14y ago
  [deleted]
- sgift 14y ago
  A K-Line is a ban of an client (or multiple clients) from an irc network: http://en.wikipedia.org/wiki/K-line_(IRC)#K-line http://en.wikipedia.org/wiki/K-line_(IRC)#K-line
- deleted 14y ago
  [deleted]
- deleted 14y ago
  [deleted]
- ahf 14y ago
  It is an IRC term for a "kill line". If your host tuple (nickname, username and hostname) somehow matches the kill line, your connection to the server will be closed with a message on why it was closed.
andor 14y ago
Unfortunately, the past few days many EFnet servers (and more are following) have had to ban an entire ISP, which has not happened in over a decade, if not longer. Naturally, something extreme must happen for this to be even considered. [... and the extreme thing is ...] the email also contained sensitive information about who this operator was, including nicknames This is ridiculous. Why would an IRC op want to keep his/her nickname from becoming public? On IRC, even IP addresses are public. Hetzners reaction seems entirely reasonable to me, especially if the server in question was "mots likely hacked".
- ranieuwe 14y ago
  What do you think happens when an IRC op gets his name passed to that kiddie? Many IRCd gets hit with 50Gbit/s all the time because of kiddie targeting operators that make such decisions.
- nknighthb 14y ago
  > Why would an IRC op want to keep his/her nickname from becoming public? That isn't what they said. > On IRC, even IP addresses are public. This may still be true on efnet. It has ceased to be true on many IRC networks. And even where it is, VPNs, shell accounts, tor, and similar methods are readily available to make the IP address worthless. > Hetzners reaction seems entirely reasonable to me Please tell us where you work so we can all be certain never to report any sort of abuse to your employer non-anonymously.
  - DangerousPie 14y ago
    > That isn't what they said. How not? The forum seems to be down now but the announcement heavily implied that all that was "leaked" was the nick name of an operator, which they claimed could somehow be used to deduce the real name and address.
    - nknighthb 14y ago
      The problem is not the public knowledge of a nick, but the information that the person using that nick made the complaint. It paints a target on a person whose nick is very often publicly associated with a real identity.
johndoeee 14y ago
I got an abuse mail from Hetzner once (they mistyped my ip). The original complaint was something autodetected by their own system, mailed to themselves. The original complaint was attached, header and all. Something about malware on an IP similar to mine. Also, when will this be in effect, the server i tested from had no problems connecting.
- arianvanp 14y ago
  I had the exact same email last week .-.
DangerousPie 14y ago
This seems like a ridiculous overreaction to me. Not only do I not see much wrong with an ISP forwarding abuse e-mails to the admin of the hacked server (who is probably a victim too), but I also find a bit hard to believe that the nickname of an operator is enough to "derive a home address".
- nknighthb 14y ago
  > I also find a bit hard to believe that the nickname of an operator is enough to "derive a home address". An IRC nick is not a mask one hides behind. It is an alternative identifier for a real person whose other personal information may be locatable.
- blibble 14y ago
  you'd be surprised as an operator of a large IRC network I am forced to keep my real name off my employer's websites, websites like github and my home address out of the public record (director's addresses and such like) it's a major nuisance
  - ulope 14y ago
    Here is an idea: use different nicknames for different things then you wont have to live in a self prescribed virtual prison
- darklajid 14y ago
  Not enough to derive a home address: That leads us back to the pseudonym discussions: Not every name that isn't on ones state-issued id is anonymous. With mine, even without clicking on my profile, you can easily find out my name and yes, my address as well. If I connect to an IRC network (mostly freenode, or gnome/gimp) I'll be online as darklajid as well. I ran a tiny irc network a couple years ago and used the same handle. For me that's not at all hard to believe.
- weinzierl 14y ago
  Maybe the op is German. In some jurisdictions it's required to give your postal address if you publish content publicly. Germany is an example, the fine is up to 50000 EUR, but the real risk is that any lawyer can send you as cease and desist in the name of consumer protection, typically this will be about 500 EUR.
h2s 14y ago
That announcement should have clarified whether the communication they sent to Hetzner explicitly requested confidentiality. If not, the incident is as much Efnets's fuckup as Hetzner's. Why did it even need to contain such sensitive personal information about the sysop in the first place?
- devicenull 14y ago
  Even if their initial email requested it, wouldn't you assume that a large provider like Hetzner has at least partially automated their abuse handling? I wouldn't count on a human actually reading it before it gets forwarded to the customer.
trotsky 14y ago
God forbid we should mildly redesign the irc semantics so that splits don't allow you to take over channels or collide clients off. I mean it's only been a big enough problem for 20 years now that you can't even host an ircd on most standard hosting contracts.
- jpswade 14y ago
  This isn't an issue with the IRC RFC. This is a deliberate choice by EFnet, it's part of it's culture and bravado. You won't see this on Freenode.
  - trotsky 14y ago
    I'm not an expert, but isn't it more accurate to say that freenodes and other networks have "patched" the flaw via chanserv and nickserv bots to mitigate the problems with the network model? I know efnet doesn't want formalized chan/nick ownership, but they clearly dont like being the world's #1 ddos sink. Why not just include a master network quorum function and a grace period under which reattached networks don't get to merge their mode changes.
    - jpswade 14y ago
      They mostly fixed the channel problems with CHANFIX. This isn't really about that though.
- xpose2000 14y ago
  Efnet has had chanfix for quite awhile now, which makes taking over channels from a split much harder, if not impossible.
  - trotsky 14y ago
    Really? Hmm. It has been 5+ years for me, my bad. But what's the point of the ddos's then? I see netflow graphs of huge attacks on major efnet servers on a regular basis.
efdee 14y ago
"One of our EFnet operators located a server that a client was using, the server was most likely hacked and/or used for illegal activities." -- so they contacted the ISP and the ISP sent their request to whoever owns the box. That sounds entirely reasonable to me. Hetzner has no idea whether or not the box owner is the suspected hacker (even EFnet assumed the server was hacked) in the same way that EFnet has no idea about whether an Hetzner employee was the suspected hacker. So three possible outcomes: (a) the box was hacked, so sending said info to the box owner was not a bad idea, (b) the box wasn't hacked, it was the box owner himself doing bad things, in which case Hetzner just gave the box owner personal information about his victim, or (c) the box wasn't hacked, and it's a Hetzner employee doing bad things, in which case EFnet just gave the employee personal information about the victim. So, exactly why are they blaming Hetzner?
- laumars 14y ago
  The issue is that Hetzner passed on personal information. It's a breach of trust.
  - efdee 14y ago
    You're missing my point though. The operator passed on personal information without knowing if a Hetzner employee might be the bad guy. Hetzner passed on personal information without knowing the server owner might be the bad guy. If the operator had contacted the server owner directly to let him know his box was hacked, the result would have been the same. Why did the operator have the expectation that an uplink of the server wouldn't just forward the mail to the server owner?
    - laumars 14y ago
      Because Hetzner is an award winning business who sells professional hosting solutions. You'd have a point if the company in question was some dodgy garage operation, but Hetzner isn't. If EFNet are going to report the issue then there has to be a trust worthy contact at some point along the chain of network ownership, and Hetzner should have been that point. Quite frankly, it's irresponsible for a company of that size to behave in that way.
JonnieCache 14y ago
Nice to see efnet retaining some of its old magic :)
TheSwordsman 14y ago
Full text: Posted on behalf of Silence, EFnet admin: Dear users, EFnet has always been a network that promotes freedom of speech. One of the core pillars of a free virtual society is trust. Trust not only amongst ourselves internally, but an undying trust in the companies that allow their users to connect to our wonderful network. We have survived over two decades, in a world that is increasingly image- and video-based. IRC can offer neither of those. IRC is based on ideas. Ideas that are exchanged in text. With text, as opposed to images and videos, one has to be put extra effort into the subliminal, the meaning, the message. This has been our catcher in the rye, and we intend to protect this content-based communication form, for as long as it is appreciated by the hundreds of thousands who every day turn to IRC for philosophical debates, dating and just about anything you can think of (I’m sure a lot of the things in that last category does not belong here in this text, but you get the picture!). We rely solely at the goodwill of others, as is the case with most things worth saving. There is no money to be made. We all do this for free. Sure, some companies might have benefited from a small level of advertisement, attracting customers to their products. But all in all, it has mostly been an uphill battle against enormous attacks, sometimes exceeding 75Gbps of DDoS. This has made it impossible for all but the largest organizations to host a server on our network, or any other large virtual society. We are Don Quijote and the weather mills are often winning. One of our key strategies is to preserve a close relationship with the major Internet- and Hosting Service Providers, as those are the networks that our users connect through. This has worked very well due to our personal involvement with a lot of said organizations. If we find an abuser on IRC, we try to not only ban him or her, but also to contact the provider so that the problem is handled at the right end, often with the involvement of law enforcement, as was the case with Kevin Mitnick, t0rn and a lot of other well publicized hackers/crackers. Unfortunately, the past few days many EFnet servers (and more are following) have had to ban an entire ISP, which has not happened in over a decade, if not longer. Naturally, something extreme must happen for this to be even considered. Almost always can we find a solution through the use of good old fashioned communication. Alas, not in this case. Well, here is the story (to the best of my knowledge): One of our EFnet operators located a server that a client was using, the server was most likely hacked and/or used for illegal activities. As IRC is often a playground for these people to use, before moving on to more serious targets (where they can make money through extortion), we take this extremely seriously. Because of the serious nature of this, our operator sent an email to Hetzner.de, a German hosting provider, to help them lower the abuse of their servers, as well as ours. This is usually a fruitful symbiotic relationships, where both parties stand to gain. However, the big difference between this case and all the other thousands of cases we have handled in the past, is that someone at Hetzner chose to forward this complaint to the actual abuser him/herself. This might seem fair enough, as anyone accused should be granted the right to defend him- or herself. However, the email also contained sensitive information about who this operator was, including nicknames (from which names can be derived, and thus, also, home addresses). We know what an impact this can have on your social, not to mention your professional life. We have seen people lose their jobs, after constant attacks and we have also seen companies lose money that is hard to fathom, considering this is still just a simple chat for friends. This is a fundamental breach of that mutual trust that has allowed us to accept clients from Hetzner to use our network - free of charge, just like we do with anyone else wanting to connect. This a give and take network, where mutual trust is vital for our survival. We are maintained by the community, and we exist solely for the community. Hetzner.de has broken one of the most fundamental aspects of any report of criminal activity or suspicion thereof; source protection. I expect us to get attacked now, which will result in a lot of work for the company kind enough to donate money and time to continue to provide us with servers, in an era where almost everything else would be more profitable. But this is an ideological problem, more than a financial one. We have been attacked before, and we will again. We are prepared. But these preparations rely on the fact that we know who the enemy is. Hetzner.de has made that impossible. As a result of this, we have decided to ban all Hetzner IP ranges (both IPv4 and IPv6) from our servers. It seems other networks are following, and I know QuakeNet has published a similar statement. We simply do not want anything to do with a company that values money over source protection and integrity. Some may argue that this was a one time mistake, and that we should not jump to conclusions so fast. Could this have been a mistake? Sure. Does it matter, given the consequences this could have had for this operator’s personal life and health? No. We do not appreciate cowards that would rather see someone else hurt, than take their responsibility. Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception. They have realized they can save money (by limiting attacks) by redirecting the attacks back at the person reporting them. That way, the hacker/cracker/kiddie using their services will not cancel their contract with Hetzner, and in return Hetzner will remain protected. Left are those of us that work for free, and who will continue to do so, for as long as there are honest, reliable companies out there, willing to go the extra mile to protect the freedom of the Internet, and, above all, freedom of press and source protection. Questions on this matter must be directed to Hetzner.de, as our involvement in this situation is over. This has been their decision based on questionable methods. It is unfortunate for them that they got caught, but it is good for the sake of the free Internet. Sincerely yours, Johan Boger, on behalf of EFnet and anyone else believing in integrity, source protection and a free Internet.
gesman 14y ago
Hetzner offer the cheapest dedicated servers with pretty hefty resources. Hence the invitation for abuse.
deleted 14y ago
[deleted]
raphman 14y ago
I have reported spammers/phishers to Hetzner in the past and experienced the same: Hetzner's default policy is to forward a complaint to the server's owner - which also kind of surprised and annoyed me. On the phone, support staff told me that Hetzner sees itself just as an uninvolved messenger between both parties. Apparently, their support ticket system automatically forwards all complaints to the server's owner without any way to opt out. The support person offered two alternatives: send anonymous complaints through a freemail service, or send the complaint to the personal address of a support team member, so that they can manually enter it into the system. Yes, this is pretty annoying.
- dsl 14y ago
  Mind sharing any team member emails you've gotten? I am still looking for a competent abuse contact at Hetzner.
- oellegaard 14y ago
  As a Hetzner customer I think this is more than fair. If someone reports abuse from my server, hetzner should forward me the report. They do not have access to the server, so why in the world would they do anything but forwarding it to the person or company that does in fact have access? Protip: Don't put your credit card number in the abuse report. Just put the logs or whatever is required to prove that there was in fact abuse from that server.
  - danudey 14y ago
    The problem is that if you have someone using Hetzner's network for spamming or other illegal activities, you'd expect the ISP to want to keep their network clean of these sorts of things and investigate. Instead, they just say 'whatever, not our problem'. On top of that, though, forwarding the complaint to the server owner just makes a mess of things, because now the server owner can target you and make your life hell. They can send you threats, DDoS your mail system, flood your inbox, etc., all the while knowing that Hetzner has their back. It's like if you saw someone assaulting someone in an alley and called the police, and the police took the information you gave them and gave it to the person who assaulted them. Now they know who you are, and they can come after you too. Now your only real solution is to just not report it when someone does bad things, because you know it's not going to get fixed and it's just likely to cause problems for you down the road.
devicenull 14y ago
Are there any actual standards for how abuse reports should be handled by service providers? What Hetzner is doing here appears to be pretty reasonable to me (it matches what companies like Level 3 and nLayer do). I can understand why you might not want sensitive email forwarded to the abuser, but why would you send that information in your initial complaint? For all you know, the person you're reporting to is the abuser.
cientifico 14y ago
This posts makes me trust more in Hetzner as a provider than the other way around. I know that if there is any problem, they will forward the problem to me as soon as possible, and they are not going to take the law into their own.
weinzierl 14y ago
"[...] someone at Hetzner chose to forward this complaint to the actual abuser him/herself. [...] Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception." I really hope this is not true. In 2011 there started to appear pornography when I searched for my name in image search. The reason was that a stackoverflow.com scraper showed my answers (which I post under my real name) but my profile picture replaced with porn. I have no idea why he replaced the profile pictures in the first place, but anyway. The scraper site was hosted at Hetzner. I phoned Hetzner. They told me to write to abuse@hetzner.de, which I did (in German). I received an auto-reply but nothing else happened. I tried to reach their phone support (09831 610061) several times during their business hours (Mo-Fr 7:30-18:00 Uhr) but no one picked up. I wrote several other mails to Hetzner but never received another reply. After about seven months the scraper went down but I have no idea if it had something to do with my complaints because I never received any feedback from Hetzner. I have know Idea who was behind the site. They could have found out about me anyway because I posted this publicly on meta.stackoverflow [1], but still: The thought that they might have learned about my identity through Hetzner is discomforting. [1] http://meta.stackoverflow.com/questions/81872/copycat-site-causes-disturbing-images-to-turn-up-in-image-search http://meta.stackoverflow.com/questions/81872/copycat-site-c...
ttrreeww 14y ago
This sounds like whining on the part of efnet...
mschuster91 14y ago
I do see why EFnet makes a drama out of this situation... check out the RIPE entry for one of their IPs: https://apps.db.ripe.net/search/query.html?searchtext=5.9.120.209&flags=&sources=RIPE_NCC&grssources=&inverse=&types=#resultsAnchor https://apps.db.ripe.net/search/query.html?searchtext=5.9.12... They do not mention that these emails get redirected to customer!

97 points by alternize 14y ago

Uhhh... that's bad. Real bad, for both sides. I can totally understand Hetzner for just forwarding abuse complaints to the client (for root servers, the hoster usually has no "emergency ssh keys"), so the faster the original owner of the server can boot out the hacker, the better. At least it's better than disconnecting the customer from the internet entirely, especially as "haxx0ring" a server is damn easy these days, given the numbers of aged Wordpress installs alone. Also, a server owner who knows his server can support me as a hacking victim better than the hoster support who often knows nothing about configuration details, OS, disk encryption etc. on the server. But I also understand EFNet, that their emails got blindly forwarded is bad, too...

OGC 14y ago
AFAIK, aged wordpress install usually run on shared hosting.
- mschuster91 14y ago
  There are quite a number of webdev/media agencies which run the Wordpress sites of their customers on their own servers, but neglect the maintenance.
claudius 14y ago
> I can totally understand Hetzner for just forwarding abuse complaints to the client (for root servers, the hoster usually has no "emergency ssh keys"), so the faster the original owner of the server can boot out the hacker, the better. Removing confidential information from such an email should take ~2min, I think that’s time well-spent, especially given that email per se is not a real-time medium.
- mschuster91 14y ago
  I guess their automated forward system regex-greps the IP out of the complaint email and then forwards the mail to the applicable customer(s), so this system actually is realtime. Also, it places the burden of dealing with spam on the customer, so that the hoster doesn't have to do spam-filtering.

codesuela 14y ago

> Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception. They have realized they can save money (by limiting attacks) by redirecting the attacks back at the person reporting them. That way, the hacker/cracker/kiddie using their services will not cancel their contract with Hetzner, and in return Hetzner will remain protected. I don't think that this has to do with Hetzner needing criminal business but rather with Hetzner not wanting to shut down an entire server if a part of it has been breached. Forwarding that complaint in it's entirety is definitely not best practice however making such allegations is neither.

nmcfarl 14y ago
That was roughly my take away. Hetzner looks really bad - but the explanation that they are incompetent and lazy sounds more likely than the criminal explanation. And EFnet looks wronged. But also looks bad for spreading allegations against Hetzner of profiteering from criminals without much more substantial backing. Sticking to the facts of the situation would have produced a more powerful, and damning condemnation.
- rhizome 14y ago
  This isn't court, I think it's reasonable to forgive humanity entering into the issue. Besides, this appears to be a citizenship issue, so banishment also comes as no surprise.
devicenull 14y ago
What are "best practices" here? I've been unable to find any guidance about how these types of issues should be handled.

subsystem 14y ago

"This has worked very well due to our personal involvement with a lot of said organizations. If we find an abuser on IRC, we try to not only ban him or her, but also to contact the provider so that the problem is handled at the right end, often with the involvement of law enforcement, as was the case with Kevin Mitnick, t0rn and a lot of other well publicized hackers/crackers." I'm not sure that is something to be proud of.

ihsw 14y ago
It is a necessary sacrifice to foster an open and mutually beneficial service.
lawnchair_larry 14y ago
Yeah, I don't recall Mitnick participating in extortion either. This author is a piece of work.
nknighthb 14y ago
Misconduct by authorities does not change the clear criminal culpability of Mitnick.
laumars 14y ago
Of course it is. If someone cost you time and money and severely harassed users of a service you're offering by illegally attacking your servers, then I bet you'd be all for reporting them as well. The problem with script kiddies is that they're too lazy to create anything of their own and thus cannot emphasis with how hard it is to set up and maintain a free public service. So they don't have any second thought about destroying other peoples hard work. Plus the pseudo-anonymity of being online and the ability to not look your victims in the eye make such crimes easy for even some of the more ethical kids to shrug off. The only way to have an open internet where everyone is free to create and contribute is to rat out those who seek to destroy it. People like the aforementioned might enjoy all the perks of an open internet, but their actions have the opposite effect. So I'm not only in favour of reporting abusers, but have actually reported attacks on my own servers to their respective ISPs.

ciupicri 14y ago

The article doesn't seem to mention if the sensitive information from the email was marked as such.

TazeTSchnitzel 14y ago
Why would it need to be? Surely Hetzner and the like would be familiar with how sensitive it could be.
- DangerousPie 14y ago
  If you consider nicknames sensitive you are gonna have to mention that, because a lot of people wouldn't.
  - TazeTSchnitzel 14y ago
    Hetzner, as a hosting provider, would likely be well aware of the workings of IRC networks, so I doubt that.

eliasmacpherson 14y ago

Doesn't explain why efnet themselves are not at fault for failing to redact the details if they are so concerned. What if the attacker was Hetzner staff?

ahnberg 14y ago
Should we perhaps have sent the request through anonymized pidgeon? When you send an e-mail to someone you can find out quite a lot of information from headers, from a nick or a given name. When contacting ISPs some people attach contact information so that the ISP can request more information, maybe even make a call to verify identity and discuss the matter to help resolve it. This doesn't mean that the information is intended for the abuser himself. I'd expect the ISP to send anonymized information/questions to the owner of said service, and not just forward "everything" (especially without discussion with the reporting party). Reporting DDoS and abusers risks putting a huge target on yourself, your company, your servers, etc. That is what the issue is about.
- eliasmacpherson 14y ago
  I specified redacting the details, i.e. the irc nickname. If the information within the headers is that sensitive, the headers should probably not be forwarded from efnet to hetzner. At its most basic their servers are attacking efnets and so should be considered hostile. By all means pass on contact information to resolve amicably - but don't leave sensitive information. There's no reason to pass on irc nicks or sensitive email addresses - what use are they to hetzner? I have the same expectation of efnet as you have for the ISP. If the issue is that efnet is being identified as the reporter to the rogue attacker, then that is not what is stated in the efnet admin's text.
  - ahnberg 14y ago
    EFnet didn't pass anything on. EFnet reported abuse, and the EFnet reporting staff members information was the information that was leaked/forwarded to the abuser. Not a third party to whom EFnet acted proxy. You seem to misunderstand the situation.

itry 14y ago

Can somebody tell me, what "kline" means? I never heard that word and dont find a dictionary entry.

dpcx 14y ago
http://en.wikipedia.org/wiki/K-line_(IRC)#K-line http://en.wikipedia.org/wiki/K-line_(IRC)#K-line
TazeTSchnitzel 14y ago
kline is short for "kill line" and refers to lines in configuration files for IRC daemons (IRCd), i.e. IP ban.
akx 14y ago
kline (K-line) basically [K]ills a network or IP block, forbidding them from connecting to the IRC server. (I think the term stems from the config files for the original ircd, circa 1988)
kenny_r 14y ago
It's IRC terminology: http://en.wikipedia.org/wiki/IRCd#K-line http://en.wikipedia.org/wiki/IRCd#K-line
deleted 14y ago
[deleted]
sgift 14y ago
A K-Line is a ban of an client (or multiple clients) from an irc network: http://en.wikipedia.org/wiki/K-line_(IRC)#K-line http://en.wikipedia.org/wiki/K-line_(IRC)#K-line
deleted 14y ago
[deleted]
deleted 14y ago
[deleted]
ahf 14y ago
It is an IRC term for a "kill line". If your host tuple (nickname, username and hostname) somehow matches the kill line, your connection to the server will be closed with a message on why it was closed.

andor 14y ago

Unfortunately, the past few days many EFnet servers (and more are following) have had to ban an entire ISP, which has not happened in over a decade, if not longer. Naturally, something extreme must happen for this to be even considered. [... and the extreme thing is ...] the email also contained sensitive information about who this operator was, including nicknames This is ridiculous. Why would an IRC op want to keep his/her nickname from becoming public? On IRC, even IP addresses are public. Hetzners reaction seems entirely reasonable to me, especially if the server in question was "mots likely hacked".

ranieuwe 14y ago
What do you think happens when an IRC op gets his name passed to that kiddie? Many IRCd gets hit with 50Gbit/s all the time because of kiddie targeting operators that make such decisions.
nknighthb 14y ago
> Why would an IRC op want to keep his/her nickname from becoming public? That isn't what they said. > On IRC, even IP addresses are public. This may still be true on efnet. It has ceased to be true on many IRC networks. And even where it is, VPNs, shell accounts, tor, and similar methods are readily available to make the IP address worthless. > Hetzners reaction seems entirely reasonable to me Please tell us where you work so we can all be certain never to report any sort of abuse to your employer non-anonymously.
- DangerousPie 14y ago
  > That isn't what they said. How not? The forum seems to be down now but the announcement heavily implied that all that was "leaked" was the nick name of an operator, which they claimed could somehow be used to deduce the real name and address.
  - nknighthb 14y ago
    The problem is not the public knowledge of a nick, but the information that the person using that nick made the complaint. It paints a target on a person whose nick is very often publicly associated with a real identity.

johndoeee 14y ago

I got an abuse mail from Hetzner once (they mistyped my ip). The original complaint was something autodetected by their own system, mailed to themselves. The original complaint was attached, header and all. Something about malware on an IP similar to mine. Also, when will this be in effect, the server i tested from had no problems connecting.

arianvanp 14y ago
I had the exact same email last week .-.

DangerousPie 14y ago

This seems like a ridiculous overreaction to me. Not only do I not see much wrong with an ISP forwarding abuse e-mails to the admin of the hacked server (who is probably a victim too), but I also find a bit hard to believe that the nickname of an operator is enough to "derive a home address".

nknighthb 14y ago
> I also find a bit hard to believe that the nickname of an operator is enough to "derive a home address". An IRC nick is not a mask one hides behind. It is an alternative identifier for a real person whose other personal information may be locatable.
blibble 14y ago
you'd be surprised as an operator of a large IRC network I am forced to keep my real name off my employer's websites, websites like github and my home address out of the public record (director's addresses and such like) it's a major nuisance
- ulope 14y ago
  Here is an idea: use different nicknames for different things then you wont have to live in a self prescribed virtual prison
darklajid 14y ago
Not enough to derive a home address: That leads us back to the pseudonym discussions: Not every name that isn't on ones state-issued id is anonymous. With mine, even without clicking on my profile, you can easily find out my name and yes, my address as well. If I connect to an IRC network (mostly freenode, or gnome/gimp) I'll be online as darklajid as well. I ran a tiny irc network a couple years ago and used the same handle. For me that's not at all hard to believe.
weinzierl 14y ago
Maybe the op is German. In some jurisdictions it's required to give your postal address if you publish content publicly. Germany is an example, the fine is up to 50000 EUR, but the real risk is that any lawyer can send you as cease and desist in the name of consumer protection, typically this will be about 500 EUR.

h2s 14y ago

That announcement should have clarified whether the communication they sent to Hetzner explicitly requested confidentiality. If not, the incident is as much Efnets's fuckup as Hetzner's. Why did it even need to contain such sensitive personal information about the sysop in the first place?

devicenull 14y ago
Even if their initial email requested it, wouldn't you assume that a large provider like Hetzner has at least partially automated their abuse handling? I wouldn't count on a human actually reading it before it gets forwarded to the customer.

trotsky 14y ago

God forbid we should mildly redesign the irc semantics so that splits don't allow you to take over channels or collide clients off. I mean it's only been a big enough problem for 20 years now that you can't even host an ircd on most standard hosting contracts.

jpswade 14y ago
This isn't an issue with the IRC RFC. This is a deliberate choice by EFnet, it's part of it's culture and bravado. You won't see this on Freenode.
- trotsky 14y ago
  I'm not an expert, but isn't it more accurate to say that freenodes and other networks have "patched" the flaw via chanserv and nickserv bots to mitigate the problems with the network model? I know efnet doesn't want formalized chan/nick ownership, but they clearly dont like being the world's #1 ddos sink. Why not just include a master network quorum function and a grace period under which reattached networks don't get to merge their mode changes.
  - jpswade 14y ago
    They mostly fixed the channel problems with CHANFIX. This isn't really about that though.
xpose2000 14y ago
Efnet has had chanfix for quite awhile now, which makes taking over channels from a split much harder, if not impossible.
- trotsky 14y ago
  Really? Hmm. It has been 5+ years for me, my bad. But what's the point of the ddos's then? I see netflow graphs of huge attacks on major efnet servers on a regular basis.

efdee 14y ago

"One of our EFnet operators located a server that a client was using, the server was most likely hacked and/or used for illegal activities." -- so they contacted the ISP and the ISP sent their request to whoever owns the box. That sounds entirely reasonable to me. Hetzner has no idea whether or not the box owner is the suspected hacker (even EFnet assumed the server was hacked) in the same way that EFnet has no idea about whether an Hetzner employee was the suspected hacker. So three possible outcomes: (a) the box was hacked, so sending said info to the box owner was not a bad idea, (b) the box wasn't hacked, it was the box owner himself doing bad things, in which case Hetzner just gave the box owner personal information about his victim, or (c) the box wasn't hacked, and it's a Hetzner employee doing bad things, in which case EFnet just gave the employee personal information about the victim. So, exactly why are they blaming Hetzner?

laumars 14y ago
The issue is that Hetzner passed on personal information. It's a breach of trust.
- efdee 14y ago
  You're missing my point though. The operator passed on personal information without knowing if a Hetzner employee might be the bad guy. Hetzner passed on personal information without knowing the server owner might be the bad guy. If the operator had contacted the server owner directly to let him know his box was hacked, the result would have been the same. Why did the operator have the expectation that an uplink of the server wouldn't just forward the mail to the server owner?
  - laumars 14y ago
    Because Hetzner is an award winning business who sells professional hosting solutions. You'd have a point if the company in question was some dodgy garage operation, but Hetzner isn't. If EFNet are going to report the issue then there has to be a trust worthy contact at some point along the chain of network ownership, and Hetzner should have been that point. Quite frankly, it's irresponsible for a company of that size to behave in that way.

JonnieCache 14y ago

Nice to see efnet retaining some of its old magic :)

TheSwordsman 14y ago

Full text: Posted on behalf of Silence, EFnet admin: Dear users, EFnet has always been a network that promotes freedom of speech. One of the core pillars of a free virtual society is trust. Trust not only amongst ourselves internally, but an undying trust in the companies that allow their users to connect to our wonderful network. We have survived over two decades, in a world that is increasingly image- and video-based. IRC can offer neither of those. IRC is based on ideas. Ideas that are exchanged in text. With text, as opposed to images and videos, one has to be put extra effort into the subliminal, the meaning, the message. This has been our catcher in the rye, and we intend to protect this content-based communication form, for as long as it is appreciated by the hundreds of thousands who every day turn to IRC for philosophical debates, dating and just about anything you can think of (I’m sure a lot of the things in that last category does not belong here in this text, but you get the picture!). We rely solely at the goodwill of others, as is the case with most things worth saving. There is no money to be made. We all do this for free. Sure, some companies might have benefited from a small level of advertisement, attracting customers to their products. But all in all, it has mostly been an uphill battle against enormous attacks, sometimes exceeding 75Gbps of DDoS. This has made it impossible for all but the largest organizations to host a server on our network, or any other large virtual society. We are Don Quijote and the weather mills are often winning. One of our key strategies is to preserve a close relationship with the major Internet- and Hosting Service Providers, as those are the networks that our users connect through. This has worked very well due to our personal involvement with a lot of said organizations. If we find an abuser on IRC, we try to not only ban him or her, but also to contact the provider so that the problem is handled at the right end, often with the involvement of law enforcement, as was the case with Kevin Mitnick, t0rn and a lot of other well publicized hackers/crackers. Unfortunately, the past few days many EFnet servers (and more are following) have had to ban an entire ISP, which has not happened in over a decade, if not longer. Naturally, something extreme must happen for this to be even considered. Almost always can we find a solution through the use of good old fashioned communication. Alas, not in this case. Well, here is the story (to the best of my knowledge): One of our EFnet operators located a server that a client was using, the server was most likely hacked and/or used for illegal activities. As IRC is often a playground for these people to use, before moving on to more serious targets (where they can make money through extortion), we take this extremely seriously. Because of the serious nature of this, our operator sent an email to Hetzner.de, a German hosting provider, to help them lower the abuse of their servers, as well as ours. This is usually a fruitful symbiotic relationships, where both parties stand to gain. However, the big difference between this case and all the other thousands of cases we have handled in the past, is that someone at Hetzner chose to forward this complaint to the actual abuser him/herself. This might seem fair enough, as anyone accused should be granted the right to defend him- or herself. However, the email also contained sensitive information about who this operator was, including nicknames (from which names can be derived, and thus, also, home addresses). We know what an impact this can have on your social, not to mention your professional life. We have seen people lose their jobs, after constant attacks and we have also seen companies lose money that is hard to fathom, considering this is still just a simple chat for friends. This is a fundamental breach of that mutual trust that has allowed us to accept clients from Hetzner to use our network - free of charge, just like we do with anyone else wanting to connect. This a give and take network, where mutual trust is vital for our survival. We are maintained by the community, and we exist solely for the community. Hetzner.de has broken one of the most fundamental aspects of any report of criminal activity or suspicion thereof; source protection. I expect us to get attacked now, which will result in a lot of work for the company kind enough to donate money and time to continue to provide us with servers, in an era where almost everything else would be more profitable. But this is an ideological problem, more than a financial one. We have been attacked before, and we will again. We are prepared. But these preparations rely on the fact that we know who the enemy is. Hetzner.de has made that impossible. As a result of this, we have decided to ban all Hetzner IP ranges (both IPv4 and IPv6) from our servers. It seems other networks are following, and I know QuakeNet has published a similar statement. We simply do not want anything to do with a company that values money over source protection and integrity. Some may argue that this was a one time mistake, and that we should not jump to conclusions so fast. Could this have been a mistake? Sure. Does it matter, given the consequences this could have had for this operator’s personal life and health? No. We do not appreciate cowards that would rather see someone else hurt, than take their responsibility. Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception. They have realized they can save money (by limiting attacks) by redirecting the attacks back at the person reporting them. That way, the hacker/cracker/kiddie using their services will not cancel their contract with Hetzner, and in return Hetzner will remain protected. Left are those of us that work for free, and who will continue to do so, for as long as there are honest, reliable companies out there, willing to go the extra mile to protect the freedom of the Internet, and, above all, freedom of press and source protection. Questions on this matter must be directed to Hetzner.de, as our involvement in this situation is over. This has been their decision based on questionable methods. It is unfortunate for them that they got caught, but it is good for the sake of the free Internet. Sincerely yours, Johan Boger, on behalf of EFnet and anyone else believing in integrity, source protection and a free Internet.

gesman 14y ago

Hetzner offer the cheapest dedicated servers with pretty hefty resources. Hence the invitation for abuse.

deleted 14y ago

[deleted]

raphman 14y ago

I have reported spammers/phishers to Hetzner in the past and experienced the same: Hetzner's default policy is to forward a complaint to the server's owner - which also kind of surprised and annoyed me. On the phone, support staff told me that Hetzner sees itself just as an uninvolved messenger between both parties. Apparently, their support ticket system automatically forwards all complaints to the server's owner without any way to opt out. The support person offered two alternatives: send anonymous complaints through a freemail service, or send the complaint to the personal address of a support team member, so that they can manually enter it into the system. Yes, this is pretty annoying.

dsl 14y ago
Mind sharing any team member emails you've gotten? I am still looking for a competent abuse contact at Hetzner.
oellegaard 14y ago
As a Hetzner customer I think this is more than fair. If someone reports abuse from my server, hetzner should forward me the report. They do not have access to the server, so why in the world would they do anything but forwarding it to the person or company that does in fact have access? Protip: Don't put your credit card number in the abuse report. Just put the logs or whatever is required to prove that there was in fact abuse from that server.
- danudey 14y ago
  The problem is that if you have someone using Hetzner's network for spamming or other illegal activities, you'd expect the ISP to want to keep their network clean of these sorts of things and investigate. Instead, they just say 'whatever, not our problem'. On top of that, though, forwarding the complaint to the server owner just makes a mess of things, because now the server owner can target you and make your life hell. They can send you threats, DDoS your mail system, flood your inbox, etc., all the while knowing that Hetzner has their back. It's like if you saw someone assaulting someone in an alley and called the police, and the police took the information you gave them and gave it to the person who assaulted them. Now they know who you are, and they can come after you too. Now your only real solution is to just not report it when someone does bad things, because you know it's not going to get fixed and it's just likely to cause problems for you down the road.

devicenull 14y ago

Are there any actual standards for how abuse reports should be handled by service providers? What Hetzner is doing here appears to be pretty reasonable to me (it matches what companies like Level 3 and nLayer do). I can understand why you might not want sensitive email forwarded to the abuser, but why would you send that information in your initial complaint? For all you know, the person you're reporting to is the abuser.

cientifico 14y ago

This posts makes me trust more in Hetzner as a provider than the other way around. I know that if there is any problem, they will forward the problem to me as soon as possible, and they are not going to take the law into their own.

weinzierl 14y ago

"[...] someone at Hetzner chose to forward this complaint to the actual abuser him/herself. [...] Unfortunately, according to trusted sources (ex-employees) of Hetzner.de, this is policy and not an exception." I really hope this is not true. In 2011 there started to appear pornography when I searched for my name in image search. The reason was that a stackoverflow.com scraper showed my answers (which I post under my real name) but my profile picture replaced with porn. I have no idea why he replaced the profile pictures in the first place, but anyway. The scraper site was hosted at Hetzner. I phoned Hetzner. They told me to write to abuse@hetzner.de, which I did (in German). I received an auto-reply but nothing else happened. I tried to reach their phone support (09831 610061) several times during their business hours (Mo-Fr 7:30-18:00 Uhr) but no one picked up. I wrote several other mails to Hetzner but never received another reply. After about seven months the scraper went down but I have no idea if it had something to do with my complaints because I never received any feedback from Hetzner. I have know Idea who was behind the site. They could have found out about me anyway because I posted this publicly on meta.stackoverflow [1], but still: The thought that they might have learned about my identity through Hetzner is discomforting. [1] http://meta.stackoverflow.com/questions/81872/copycat-site-causes-disturbing-images-to-turn-up-in-image-search http://meta.stackoverflow.com/questions/81872/copycat-site-c...

ttrreeww 14y ago

This sounds like whining on the part of efnet...

mschuster91 14y ago

I do see why EFnet makes a drama out of this situation... check out the RIPE entry for one of their IPs: https://apps.db.ripe.net/search/query.html?searchtext=5.9.120.209&flags=&sources=RIPE_NCC&grssources=&inverse=&types=#resultsAnchor https://apps.db.ripe.net/search/query.html?searchtext=5.9.12... They do not mention that these emails get redirected to customer!