30 ms·

DDoS Attack Against Dyn Managed DNS

• meshko 10y ago
  Very funny guys, can you stop now? We have a demo in 4 minutes.
  • kiostech 10y ago
    Maybe run in localhost bro
    • ChadMoran 10y ago
      Or bra
      • cloudjacker 10y ago
        Exhibit A of MEN having zero consequences for their sexual assault on the English language! Especially in the tech industry, where English sexual assault is unlike other industries!
  • iagooar 10y ago
    How did the demo go?
    • hueving 10y ago
      Like all live demos.
      • icpmacdo 10y ago
        my condolences
    • meshko 10y ago
      Well, the parts that relied out outside services hooked up via SSO were not demoed, but majority of it worked fine because demo server was misconfigured to not actually rely on the external services. It is pretty funny.
      • praneshp 10y ago
        It's Friday. Story/short write up appreciated
        • meshko 10y ago
          I am a bit paranoid about disclosing details, but basically our SAML IDP was down, so the sales person couldn't log in at all. I was messing with the demo server to convince myself that it is 100% IDPs fault and we can't do anything about it, and discovered to my surprise that the form-based authentication was not disabled on it (normally our servers are in one mode or the other, but not both, even though this is an artificial separation). So I gave them the direct link to the form based entry point and most of the demo could be done.
    • deleted 10y ago
      [deleted]
  • erowtom 10y ago
    We had a demo at the exact same time. (Internal weekly product demo, not that critical). We did it on local host, the only one reliable 100% of the time.
  • maherbeg 10y ago
    I find having a demo video backup is always a good idea.
  • mudge 10y ago
    I had a client ready to pay. Freshbooks was down.
• adamrights 10y ago
  We were affected @WSJ as well.
• _ar7 10y ago
  Almost every website I visit except HN seems to be down...
  • chrisabrams 10y ago
    Same here, that's odd isn't it ;)
    • pyvpx 10y ago
      well, first thing one does to use CloudFlare is migrate to their DNS. A CF-hosted site isn't going to be using Dyn...
• tbarbugli 10y ago
  Github does not work for 100% the time
  • agotterer 10y ago
    GitHub is working for me from NYC.
    • geekamongus 10y ago
      It's down in NC.
  • bananicorn 10y ago
    Weird, works for me - from Italy (not sure if there isn't just some caching going somewhere down the line and I can see it because of that) edit: nevermind, it's almost certain i've got it cached
    • owenwil 10y ago
      Definitely a DNS cache on your computer (Or even in Chrome)
      • artursapek 10y ago
        It could also be geographic. Their update did say "some customers". I'm in New York and I'm also seeing the outage.
      • detaro 10y ago
        I can query the authoritative ns*.p16.dynect.com DNS servers from Europe (Germany in my case), and the traceroute looks like it's near Frankfurt. So the anycasted copies here seem fine.
        • detaro 10y ago
          and now these seem down as well. EDIT: and up again half an hour later
• jtmarmon 10y ago
  I'm updating a list of confirmed outages as I see them here https://news.ycombinator.com/item?id=12759520 https://news.ycombinator.com/item?id=12759520 So far twitter, etsy, soundcloud, spotify, github, pagerduty...crazy that this can even happen
  • owenwil 10y ago
    Yep! Intercom is down as well
  • olegkikin 10y ago
    They all work fine for me.
    • owenwil 10y ago
      Caching is a magical thing :)
      • dspillett 10y ago
        If anycast routing is in play, which is not unlikely with a DNS service like that, then it may also be that specific servers are being attacked so the outages don't affect users in all locations as some will be routed to infrastructure that is not affected.
    • mcjiggerlog 10y ago
      Me too. I'm in Spain - definitely seems to be geographical.
  • colanderman 10y ago
    I'm surprised; I would have thought such large sites would use more than one DNS provider? I mean: $ host -t NS twitter.com twitter.com name server ns4.p34.dynect.net. twitter.com name server ns3.p34.dynect.net. twitter.com name server ns2.p34.dynect.net. twitter.com name server ns1.p34.dynect.net. I would have expected at least one of those to be somewhere else. What is the reason they would not have a backup provider?
• deleted 10y ago
  [deleted]
• dudul 10y ago
  Damn, I've spent the past 30 minutes trying to update my DNS and playing with my router config! :) No GitHub, well, it's gonna be a fun Friday...
  • piquadrat 10y ago
    putting something like 192.30.253.113 github.com into your /etc/hosts (or other appropriate location for your OS) should get you going again.
    • geekamongus 10y ago
      Careful...some people may be trying to get out of work today.
    • profmonocle 10y ago
      Just remember to take it out when this is resolved. IPs shuffle around all the time, and that IP might not always work for the main github.com domain.
  • deleted 10y ago
    [deleted]
  • 6d6b73 10y ago
    Isn't it the whole idea of git to make user independent from place like github? After all if you have the repository on your local machine you can continue working as if nothing happened, right?
    • dudul 10y ago
      Build server (e.g. CircleCI, Travis, etc), reviewing/merging pull requests, accessing 3rd party library READMEs. You know, doing software stuff.
      • 6d6b73 10y ago
        Most of this stuff you should have locally, and not depend on 3rd party websites.
• dudul 10y ago
  And there is no twitter to tweet about it!!!
• wnm 10y ago
  Heroku also seems to be affected. I'm getting this when I run 'heroku status': >> We are seeing a widespread DNS issue affecting connections to our services both internally and externally.
• artursapek 10y ago
  Twitter, Reddit, wow. I was so confused for a moment. Thankfully HN is here to explain.
  • tunap 10y ago
    I had several, sporadic 'secure connection could not be established' yesterday while trying to open HN, amongst others. Painfully slow page load times across the board, too(Craigslist, Monoprice,weather.gov, etc) Still may be my buggy phone SIM...
    • dlg1416 10y ago
      wait, buggy phone sims is a thing?
      • tunap 10y ago
        Sorta. When I changed phones I cut my micro SIM down to nano size. Cut a wee bit too much off and it now can slide off contacts if jarred... gotta get a new SIM.
• deleted 10y ago
  [deleted]
• mjpa 10y ago
  Is it really an internet wide outage? Only 2 of the points in the US are affected on https://www.whatsmydns.net/ https://www.whatsmydns.net/ for the domains we've got on Dyn - same for Twitter etc
  • owenwil 10y ago
    If it's under a denial-of-service it's possible that it may respond correctly part of the time.
    • toast0 10y ago
      Since many (all?) of Dyn's authoritative server IPs are anycast, attack traffic is probably not well distributed either. If you're routed to a server that's getting a lot of attack traffic, you're likely to have problems, but a server without much attack traffic will work fine.
  • geekamongus 10y ago
    "Widespread" might be a better term.
• scrollaway 10y ago
  Relevant (or at least a-propos) post by Bruce Schneier, from a month ago: "Someone Is Learning How to Take Down the Internet" https://www.schneier.com/blog/archives/2016/09/someone_is_lear.html https://www.schneier.com/blog/archives/2016/09/someone_is_le... Edit: And to be clear: I don't mean to imply there's any connection :)
  • jgrahamc 10y ago
    Really? It's unclear there's any connection between that vague article and Dyn's problems today.
    • FartyBumMcPoops 10y ago
      http://dyn.com/blog/post-mortem-attack-to-dyn-standard-dns-nameservers/ http://dyn.com/blog/post-mortem-attack-to-dyn-standard-dns-n...
      • ljosa 10y ago
        Let's hope they update that page after today's incident.
  • curiousgal 10y ago
    >We don't know who is doing this, but it feels like a large nation state. China or Russia would be my first guesses. Why not the USA?
    • manojlds 10y ago
      With the elections coming, USA would be on the top of my list.
      • guccifer42 10y ago
        Good afternoon, Colonel Ivanoff.
      • 10y ago
• CarVac 10y ago
  Using Google Public DNS fixed things for me.
  • ssebastianj 10y ago
    I'm using Google Public DNS too. I don't really know if there is a relation with DynDNS but I'm still experiencing issues in GitHub and Twitter, like partial loading of images.
    • dichardson 10y ago
      Perhaps Google had old (but valid) records still in their cache for a while. Google DNS was working for me for a while, and then stopped. Apparently Dyn has the problem fixed, but maybe there is some TTL based propagation delays still. I updated my internal network to use Dyn's internet guide/public DNS and the problem is fixed. Maybe this is their strategy: we break it, you buy it ;) https://help.dyn.com/internet-guide-setup/ https://help.dyn.com/internet-guide-setup/ If you can't load that page, the public DNS servers are: 216.146.35.35, 216.146.36.36
• zappo2938 10y ago
  Explains why the Heroku API is down.
• jread 10y ago
  Seems to be impacting POPs in US East most severly. We use Ripe Atlas to assess the impact of DNS outages, and in the past hour have measured about 50-60% recursive query failure from a few hundred probes in that region: https://cloudharmony.com/status-for-dyn https://cloudharmony.com/status-for-dyn
  • jread 10y ago
    Now impacting multiple regions including both US East and US West with very query failure ratios - 50-70%
• deleted 10y ago
  [deleted]
• Legogris 10y ago
  Microsoft's visualstudio.com's build servers fail to resolve Github and New Relic. So much for my Friday night deploy to staging.
• cultavix 10y ago
  Not sure if related but circleci.com is down for us do to a "DNS issue" !
  • jessemillar 10y ago
    Definitely related. Can confirm.
• altyus 10y ago
  For me redirecting my DNS to Google public DNS 8.8.8.8 and 8.8.4.4 did the trick.
  • mbym 10y ago
    Same story for me here today (reporting from Cork, Ireland)
    • dankohn1 10y ago
      I'm a Verizon FIOS customer in NYC and was unable to reach nytimes.com and several other sites this morning. Switching my DNS to Google's (8.8.8.8 and 8.8.4.4) seemed to fix the problem, but I don't understand why yet.
  • bpicolo 10y ago
    I was on Google dns until I joined my VPN a bit ago. The public DNS was failing to hit lots of things. (NY, USA)
  • dijit 10y ago
    that's not going to help much if the authoritative name servers (which is what dyn is, btw) go down for more than a day. Max record cache time is 86400s (24h), so if the attackers can keep it down for 24h then google will have to have custom instructions in place (or cache more aggressively than the RFC allows)
    • wongarsu 10y ago
      Since the attacked dyndns DNS servers are evidently anycast, the google server you are reaching might connect to a different dyndns server than you do. If google has luck to reach a less overloaded server, they might get an answer where you get none.
      • TurningCanadian 10y ago
        Side note: In addition, Google Public DNS engineers have proposed a technical solution called EDNS Client Subnet. This proposal allows resolvers to pass in part of the client's IP address (the first 24/64 bits or less for IPv4/IPv6 respectively) as the source IP in the DNS message, so that name servers can return optimized results based on the user's location rather than that of the resolver. To date, we have deployed an implementation of the proposal for many large CDNs (including Akamai) and Google properties. The majority of geo-sensitive domain names are already covered. from https://developers.google.com/speed/public-dns/faq https://developers.google.com/speed/public-dns/faq
• alexmorenodev 10y ago
  Here in Brazil things are pretty slow. "Oh, maybe its our shitty ISP screwing up everything again." No, it's in a bigger scale.
• sly010 10y ago
  I am confused. Are so many big websites using Dyn, or does Dyn have some special role in the DNS chain in the US?
  • aeden 10y ago
    Dyn has no special role, they just provide DNS for a large number of high-traffic sites and services.
  • ljosa 10y ago
    They're widely used because they were one of the few providers of geo-aware DNS service for a long time. (These days there are other, cheaper options, including Amazon Route53.)
    • planetix 10y ago
      Other option would have been Akamai's geo-aware DNS that has been available for a long time.
      • corobo 10y ago
        If anyone else is like me they assume Akamai = Expensive.
        • joncrane 10y ago
          and they would be right.
    • RijilV 10y ago
      It's pretty wild that AWS is having impacts for the dyn outage given the shoutout to Route53. Wonder what they know that they don't?
      • nixgeek 10y ago
        Not sure if people are confusing Amazon with AWS in this case. $ dig +short ns amazon.com ns1.p31.dynect.net. pdns1.ultradns.net. ns4.p31.dynect.net. pdns6.ultradns.co.uk. ns3.p31.dynect.net. ns2.p31.dynect.net.
• ilostmykeys 10y ago
  The Internet is so resilient. LOLz.
  • wongarsu 10y ago
    The internet is resilient against being completely taken down (as demonstrated by everything working just fine for me from Germany). It's explicitly not resilient against taking parts of it down. You can take some continents off the internet entirely by cutting half a dozend cables, but it's extremely hard to make the internet unusable for everyone.
• RRRA 10y ago
  Those distributed alternatives look better everyday... if only there was a working group and a transitional path.
• ljosa 10y ago
  AWS says "We are investigating elevated errors resolving the DNS hostnames used to access some AWS services in the US-EAST-1 Region." Is that coincidental, or are they being DDoSed also?
  • jpdlla 10y ago
    Have some errors logged on our side of problems resolving DNS hostname for SES email-smtp.us-east-1.amazonaws.com
  • cyberferret 10y ago
    That might explain why we are down - most of our EC2 instances are in us-east-1. Looks like Amazon SQS is impacted too. We are getting a stream of undeliverable messages, and our 'dead letter' queue is filling up!
  • colanderman 10y ago
    Apparently us-east-1 is backed by Dyn (and only Dyn) as well? $ host -t NS us-east-1.amazonaws.com us-east-1.amazonaws.com name server ns3.p31.dynect.net. us-east-1.amazonaws.com name server ns1.p31.dynect.net. us-east-1.amazonaws.com name server ns2.p31.dynect.net. us-east-1.amazonaws.com name server ns4.p31.dynect.net. That's… utterly bizarre to me. us-east-2 has a more diverse selection: $ host -t NS us-east-2.amazonaws.com us-east-2.amazonaws.com name server u4.amazonaws.com. us-east-2.amazonaws.com name server u6.amazonaws.com. us-east-2.amazonaws.com name server u3.amazonaws.com. us-east-2.amazonaws.com name server u2.amazonaws.com. us-east-2.amazonaws.com name server u1.amazonaws.com. us-east-2.amazonaws.com name server u5.amazonaws.com. us-east-2.amazonaws.com name server ns2.p31.dynect.net. us-east-2.amazonaws.com name server ns1.p31.dynect.net. us-east-2.amazonaws.com name server pdns1.ultradns.net. us-east-2.amazonaws.com name server pdns5.ultradns.info. us-east-2.amazonaws.com name server ns3.p31.dynect.net. us-east-2.amazonaws.com name server ns4.p31.dynect.net. us-east-2.amazonaws.com name server pdns3.ultradns.org. Not that anyone should be running a service whose availability they care about solely in us-east-1 anyway…
    • marcoslater 10y ago
• piker 10y ago
  CNN.com is knocked out by this attack as well. I could see that as a useful target.
• pmuk 10y ago
  I'm getting DNS errors on my PS4 when trying to download stuff, I guess it's related!
  • pmuk 10y ago
    Switching to Google's public DNS seems to have fixed it!
• staki3647 10y ago
  It is a false flag. They are trying to pin it on Russia so they can start a war.
• jssjr 10y ago
  I'm a GitHub employee and want to let everyone know we're aware of the problems this incident is causing and are actively working to mitigate the impact. "A global event is affecting an upstream DNS provider. GitHub services may be intermittently available at this time." is the content from our latest status update on Twitter (https://twitter.com/githubstatus/status/789452827269664769 https://twitter.com/githubstatus/status/789452827269664769). Reposted here since some people are having problems resolving Twitter domains as well.
  • afshinmeh 10y ago
    Just being curious, why don't you use different DNS servers?
    • pcai 10y ago
      Most services, even if they aren't the size of Github, can't change their DNS provider on a dime.
      • colanderman 10y ago
        It's not a question of switching; you can host your DNS records at multiple providers.
        • afshinmeh 10y ago
          yup, that's what I meant. they can use different DNS providers, e.g. route53 AND dyn
          • alberts00 10y ago
            Route53 doesn't allow using it as slave DNS. https://forums.aws.amazon.com/thread.jspa?threadID=56011 https://forums.aws.amazon.com/thread.jspa?threadID=56011
            • 10y ago
• staki3647 10y ago
  It is a false flag. Our govt is trying to use it as an excuse to start a war with Russia.
• deleted 10y ago
  [deleted]
• jtmarmon 10y ago
  Semi related: I noticed this incident right when it began, but not because I was trying to access a website. This started happening to me: http://imgur.com/PPlaY5o http://imgur.com/PPlaY5o Then when I went to push to github out of fear my computer was about to soil itself, that failed too, and I noticed the outage. Does anyone know if the above errors could be related to the outage? I'm using vim inside tmux with zsh as my shell. Maybe zsh does some kind of communication with gh while running? I restarted my computer and it's still happening
  • scrollaway 10y ago
    The zsh default git plugin definitely doesn't touch github, or the network in general. Are you using some oh-my-zsh github plugin by any chance?
    • jtmarmon 10y ago
      plugins=(git rbenv nvm gitfast zsh-autosuggestions github)
      • scrollaway 10y ago
        So, yes. https://github.com/robbyrussell/oh-my-zsh/wiki/Plugins#github https://github.com/robbyrussell/oh-my-zsh/wiki/Plugins#githu... oh-my-zsh is, imho, way overengineered and bloated. Leads to all sorts of issues like the one you're encountering there. I would recommend sticking to a plain zshrc file that you can read, edit and fully understand. The one I wrote and am using day to day is available here, with documentation: https://github.com/jleclanche/dotfiles https://github.com/jleclanche/dotfiles
        • jtmarmon 10y ago
          removed the github plugin, reloaded zsh and happened again 5 min later. I believe it has to do with slack, because the issue resolved itself after closing. maybe slack got pwned and all slack users are being used as part of the botnet lol
• cyberferret 10y ago
  Hmm... Seems to be quite widespread. Some of our Amazon AWS services (located in the US) that rely on SQS are reporting critical errors. Intercom.io is also down at present, which we use for support for our web apps. Not looking very good from here (in Australia).
• deleted 10y ago
  [deleted]
  • deleted 10y ago
    [deleted]
• d--b 10y ago
  Looks like at least some of it is resolved. spotify is back
• im3w1l 10y ago
  Fascinating weak spot!
• r1ch 10y ago
  Surprised to see so many big names relying on a single provider. DNS is designed to be distributed, it should be possible to avoid a single point of failure.
  • mayli 10y ago
    This question came to my mind when I saw this post. The possibility might be the management cost. Synchronize between different providers can go wrong and might hard to debug when end users get different replies.
• mdtancsa 10y ago
  Anyone know any details of what the attack looks like ? I had a quick look in my (albeit small) network to look for odd flows going to their ASN33517, but didnt see much that looked odd on first glance...
• foobarbecue 10y ago
  According to Fortune, Hacker News "reported" on the incident. Are we journalists now? "Popular tech site Hacker News reported many other sites were affected including Etsy, Spotify, Github, Soundcloud, and Heroku." -- http://fortune.com/2016/10/21/internet-outages/ http://fortune.com/2016/10/21/internet-outages/
  • mxuribe 10y ago
    No, not necessarily journalists; rahter, an information source...Fortune - a site/company known for journalism/reporting - now just gave HackerNews more legitimacy as an official information source...Now with this power, please use it responsibly. ;-)
    • stingraycharles 10y ago
      Too bad that the majority of the readers will think that HackerNews is somehow related to the "Hackers" that took down the internet.
      • figgis 10y ago
        Eh the name implies (at least in that context) this website would be used to keep track of the hackers.
  • Raphmedia 10y ago
    No but we are a hivemind of smart individuals that correctly upvote important information and downvote irrelevant information. Most of the time, you can be sure that the top HN listings are going to be relevant.
  • melvinram 10y ago
    I wonder if they used "Hacker News" as the source on that because it contains the word "hacker" and they wanted to say "oh, hackers read this site"... as in "the blackhat, break into stuff, steal your money, deface your website" folks.
  • pocoforte 10y ago
• nbrempel 10y ago
  It's a strange coincidence that Hover DNS was down for same reason a week ago. http://hoverstatus.com http://hoverstatus.com
• mirekrusin 10y ago
  They should do it once a year and call it Friday without Internet Day.
  • Asparagirl 10y ago
    Better yet, have one day a year that is "Red Team Day" where people hunt for vulnerabilities so that assessments can be done, and companies can later fix any issues noted. Like how earlier this week there was a statewide earthquake drill in California, local emergency sirens were sounded, schoolkids practiced hiding under desks, etc. The Internet needs periodic tests like that too.
    • jessemillar 10y ago
      I absolutely love this idea! Would be a bit tricky to implement, but would definitely improve security in the long run.
• metaverse 10y ago
  While my app isn't resolved using DYN, we are relying on APIs on our EC2 backend that use their DNS. Is there a Linux DNS caching server that will serve from a local cache primarily, and do lookups in the background instead to update the local cache? During the period DYN was down, it would've continued severing from the local cache and retried the background lookups, keeping my app up. I can also see it improving performance as my servers currently do lookups to the EC2 DNS on each http request...
  • nixgeek 10y ago
    If you're in us-east-1 then you potentially do actually rely on Dyn even for the amazonaws.com instance hostnames. https://gist.github.com/agh/4e20df0d2d3bfa189477569b77f72e24 https://gist.github.com/agh/4e20df0d2d3bfa189477569b77f72e24
    • metaverse 10y ago
      Seems then that ELB has a local cache because http requests were reaching my app servers throughout the outage.
• CodeSheikh 10y ago
  Let's assume, that foreign countries such as Russia or China would be trying to sabotage our elections on Nov 8th night. What are the severe economic and political backlash that we can deal with if we cut off the traffic coming in from those region (not in a "we control the internet" kinda way)? I am sure they already have nodes operating within the USA. A lot of major tech companies use CDNs that can still serve traffic globally to the consumers of those countries. Even better, how about we regulate and slow down all of incoming traffic for say half day on election day? Is it even possible?
  • transfire 10y ago
    But then why would they be doing it right now? I'm sure they already know if they can do it or not. I don't think they need to do a large scale test run that would put people on high alert. They'd keep their head down until election day. But then, what is China or Russia going to get out of doing something like this? It isn't going to change anything. Hillary is the next president regardless. Hell, even if no votes could be counted I am sure the Supreme Court wouldn't have a problem calling it for her. So to me the idea that China and Russia is doing this for political reasons doesn't make any sense.
• throwaway1021 10y ago
  I'm wondering if this has anything to do with them hiring a new CEO, Colin Doherty. I worked for him and he sucks.
• ohblahitsme 10y ago
  Twitter and Github are still down here in LA (and confirmed on isup.me)
  • jessemillar 10y ago
    Both are down here in Utah too.
    • flowless 10y ago
      New attack incoming https://www.dynstatus.com/incidents/nlr4yrr162t8 https://www.dynstatus.com/incidents/nlr4yrr162t8
    • trusche 10y ago
      Same from Ireland.
  • runn1ng 10y ago
    They were up most of the day in here (Prague, Czech Republic, Europe), but it's down now (started about 20 minutes ago). It seems to be another wave of the attack.
    • stephankoelle 10y ago
      Can confirm, europe was up until then
  • elwell 10y ago
    and mixpanel and zendesk
  • elwell 10y ago
    https://www.whatsmydns.net/#A/twitter.com https://www.whatsmydns.net/#A/twitter.com
• devnull42 10y ago
  Dyn reporting another attack started at 15:52 UTC.
• marcoslater 10y ago
  It seems to have just gone again. This time everywhere, not just US-EAST. O.o Edit: Seems its hitting their anycasted ranges and taking everything off with it. Even their status page appears to be offline from here.
• raemike123 10y ago
  USA cyber defenses are NOT up to the task of defending our critical electronic infrastructure. Letting every company that runs critical services decide their own security posture is not scalable and has left us vulnerable. While no one is getting hurt, we are taking cyber missile hits from our enemies and eventually the damage will be worse. Other countries with more central controls will be less vulnerable than we are to crippling infrastructure take downs.
  • bluejekyll 10y ago
    No. What we need are new techniques for creating back-pressure to all the routers which are forwarding on this type of attack. The issue is that our Routing technology does not give downstream nodes any way to push back on the flood of packets. Cisco could step up to the plate here. And no, I'm not talking about firewalls. We need newer ICMP type packets to create this back-pressure, so that we can stop floods like this.
    • andrewmchen 10y ago
      It looks like we used to have something similar but it got deprecated. https://tools.ietf.org/html/rfc6633 https://tools.ietf.org/html/rfc6633
      • Animats 10y ago
        I was a big fan of ICMP Source Quench in the early 1980s, but it wouldn't help now. It doesn't have authentication.
    • bogomipz 10y ago
      >"What we need are new techniques for creating back-pressure to all the routers which are forwarding on this type of attack" What is this type of attack? TCP/UDP/ICMP has no notion of of a compromised host. Or even that it was crafted packet. Back pressure already exists in TCP, see slow-start and window sizes, flow control is part of the "control part". When a router's porst buffers are full the router drops the packets on the floor. It does not do further processing of those packets. What would ICMP do here? If I have a million compromised hosts and each sends a single SYN packet towards a destination host, how would ICMP help? ">Cisco could step up to the plate here" What would Cisco do? Cisco doesn't control the ICMP protocol. I think you are not understanding ICMP. The job of ICMP is to report error conditions. ICMP serves as a helper to IP which is itself unreliable and has no form of error control or checking. A router or host being overrun is not a network error condition it is a resource condition. No ICMP type is ever going to be able to stop a host from originating UDP/TCP/ICMP towards a destination. Even if it could you would just overwhelm it in the outbound direction by replying to potentially millions of hosts.
• eredi93 10y ago
  and the attacker are back. DDoS v2 is here
• middleman90 10y ago
  Shopify is down
• chromaton 10y ago
  I can't currently get resolution on www.paypal.com. $ dig @8.8.8.8 www.paypal.com ; <<>> DiG 9.8.1-P1 <<>> @8.8.8.8 www.paypal.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 17925 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.paypal.com. IN A ;; Query time: 29 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Fri Oct 21 12:35:33 2016 ;; MSG SIZE rcvd: 32
  • robbyt 10y ago
    Me either. I'm in NYC on TWC. I can't resolve PayPal with Google's DNS or with TWC dns servers. Also, I'm unable to resolve Twitter on TWC dns.
    • deleted 10y ago
      [deleted]
  • heyts 10y ago
    I'm in New-York too and can't resolve Paypal, Etsy, Soundcloud, Github, Netflix, Heroku or Twitter
    • devy 10y ago
      I'm in NYC too. Github.com is resolving/working fine. Netflix.com is resolved but all assets (probably) weren't loading. Additionally Zendesk is also affected.
      • misiti3780 10y ago
        github is down for me in LES
        • tekromancr 10y ago
          I'm in midtown on TWC. All the things mentioned are down.
• cm3 10y ago
  Just to be clear, this is a DDoS against Dynect's NS hosts, right? I'm confused because of the use of "dyn dns", which to me means dns for hosts that don't have static ip addresses. I'm actually surprised so many big-name sites rely on Dynect, which I hadn't heard of, but more importantly don't seem to use someone else's NS hosts as 2nd or 4th entries.
  • detaro 10y ago
    The company is just called "Dyn", DynECT is a product name, but yes.
    • cm3 10y ago
      Thanks, I was really confused when I read the title "Massive Dyn DNS outage" and how that affects Twitter or Github.
• atsidi 10y ago
  I've been having the same problem accessing github in particular. Just for fun, I opened the Opera browser and activated the built-in VPN. That got everything going again. At least for browsing, not so useful for my git pulls and pushes.
• andmarios 10y ago
  OpenDNS servers seem the only ones that still work. Kudos. It may not be the proper action but this kind of soft-fail scenario (use the old DNS until you can contact the DNS servers and get new ones) is much better. echo "nameserver 208.67.222.222" | sudo tee -a /etc/resolv.conf
• adobrawy 10y ago
  Twitter and GitHub is down on Scaleway (AS12876) and Tiktalik (Warsaw, Poland, Europe, AS198717) network too (no response from dynect.net).
• mirekrusin 10y ago
  Github doesn't work again for me :(
• danyork 10y ago
  https://cloudharmony.com/status-for-dyn https://cloudharmony.com/status-for-dyn is now (12:43pm EDT) showing Dyn's "US East" and "US West" centers as being down. Anyone know anything about this Cloudharmony service? How often does it update? and what is it monitoring?
• llamataboot 10y ago
  Heroku is still having problems as well
• emmet 10y ago
  Is it just me or are these kind of attacks becoming way more frequent recently? This kind of widespread outage seems so new, but again, that might just be me.
• bhauer 10y ago
  Out of curiosity, why do caching DNS resolvers, such as the DNS resolver I run on my home network, not provide an option to retain last-known-good resolutions beyond the authority-provided time to live? In such a configuration, after the TTL expiration, the resolver would attempt to refresh from the authority/upstream provider, but if that attempt fails, the response would be a more graceful failure of returning a last-known-good resolution (perhaps with a flag). This behavior would continue until an administrator-specified and potentially quite generous maximum TTL expires, after which nodes would finally see resolution failing outright. Ideally, then, the local resolvers of the nodes and/or the UIs of applications could detect the last-known-good flag on resolution and present a UI to users ("DNS authority for this domain is unresponsive; you are visiting a last-known-good IP provided by a resolution from 8 hours ago."). But that would be a nicety, and not strictly necessary. Is there a spectacular downside to doing so? Since the last-known-good resolution would only be used if a TTL-specified refresh failed, I don't see much downside.
  • drinchev 10y ago
    All will break the moment that one of the websites that you access makes a server-side request to another website ( think about logging-services, server-clusters, database servers, etc - they all either have IPs or most-likely some domains. )
    • bhauer 10y ago
      I'm not sure I understand what you're saying. The scenario is that my local network's caching DNS resolver retains resolutions beyond the authority-provided TTL in the event that a TTL-specified refresh at expiration fails. Therefore, my web browser may—in the very rare situation where this arises—make an HTTP request to an IP address of a server that has been intentionally moved by a service provider (let's assume they did so expecting their authoritative TTL to have expired). Since this scenario only arises because my caching resolver wasn't able to reach the authority, I'm not seeing a downside. But if I understand your reply correctly, you are saying that the web server I've contacted may, in turn, be using a DNS resolver that is similarly configured to provide last-known-good resolution when its upstream provider/authority cannot provide resolution. This would potentially result in that web server making an HTTPS API request to a wrong IP, again only in the rare case where we have defaulted to a last-known-good resolution. I'm not really seeing the problem here except that the HTTPS request might fail (if the expected service was moved and no longer exists at the last-known-good IP), but how is that worse than the DNS resolution having failed? In both cases, the back-end service request fails.
• devy 10y ago
  Is Zendesk being affected? Their status page is reporting external DNS provider is having DNS issue [1] and most of their sites are being affected. [1] https://status.zendesk.com/ https://status.zendesk.com/
  • jorts 10y ago
    Yes, they were affected.
• darkmouth 10y ago
  and its down again
• deleted 10y ago
  [deleted]
• transfire 10y ago
  Oo oo, I know! Iran did it!
• chatmasta 10y ago
  I'd like to see proof of this attack from an outside network observer. Is it possible the government could force a DNS provider to pretend to fall victim to a DDoS attack, as a form of a false flag cyber attack?
• transfire 10y ago
  Must be trying to stop the latest Julian Assange leak.
  • alasdair_ 10y ago
    The Wikileaks twitter feed is putting out some really weird stuff lately. They are claiming their "supporters" are behind the attacks, which makes little sense.
• leesalminen 10y ago
  Boulder here. Can't resolve Wufoo or PayPal using 8.8.8.8
• patmcguire 10y ago
  Any quick script to see if a given domain ultimately resolves to them? My SaaS company has a lot of custom domains from whatever DNS servers pointed at us and I'd like to be able to tell people whether it's our fault or not.
  • stevekemp 10y ago
    `whois $domain`. But it should be "obvious" if your users report "Server not found" vs. "Cannot connect" or "Page not found" style errors.
  • mdavidn 10y ago
    `dig NS $domain` Query for the root domain, without any subdomains like www. That is, you need to check the "zone apex," the shortest name purchased from a registrar and potentially delegated to Dyn. Look for dynect.net in the list of authoritative name servers.
    • patmcguire 10y ago
      Yeah, I tried that. I don't see dnyect in a lot of domains that are failing, and it's clearly related somehow, they didn't all break at the same time by coincidence.
• deleted 10y ago
  [deleted]
• kilroy123 10y ago
  Interesting. Lots of sites have been down for me, here in Mexico City. Twitter. Github. Loads of other random sites. When I turned on my US based VPN. It all started working again.
• Raed667 10y ago
  You can add Netflix to the list. GET https://art-s.nflximg.net net::ERR_NAME_RESOLUTION_FAILED GET https://assets.nflxext.com net::ERR_NAME_RESOLUTION_FAILED
• BlackGuyCoding 10y ago
  Anyone having any issues with WhatsApp? Mobile text seems to work fine but all images fail, Desktop & web browser aren't connecting at the moment (west coast)
• Supersam654 10y ago
  OpenDNS DNS Servers (208.67.222.222 and 208.67.220.220) are still resolving websites while my typical fallback to 8.8.8.8 is not.
  • foxhop 10y ago
    I noticed the same pattern.
• nodesocket 10y ago
  I've been singing the praise of AWS Route53 for a long time, they up and running. I can't believe major multi-million dollar companies (Twitter, GitHub, Soundcloud, Pagerduty) would not run a mix of multiple DNS providers. Also what is happening is a cascade effect, where a 3rd party being down effects others.
  • crftr 10y ago
    > I've been singing the praise of AWS Route53 for a long time, they up and running. I'm a fan of Route53, too. But can we say that it weathered the attack? Or was it just lucky that its systems weren't targeted?
    • cloudjacker 10y ago
      > Or was it just lucky that its systems weren't targeted? I was wondering that too
    • stevekemp 10y ago
      One of the reasons why Route53 is good is because they give different nameservers to each hosted zone - unless you choose to use a branded record-set. I've seen them be hit by dDos attacks in the past, but never had any significant impact. (I wrap Route53 and handle storing DNS records in a git repository over at https://dns-api.com/ https://dns-api.com/ Adding support for other backends is my current priority to allow more redundancy.)
• newsat13 10y ago
  Switch to OpenDNS servers - 208.67.222.222 and 208.67.220.220. Even google NS are down it seems. Heroku works after switching to opendns.
  • bitskits 10y ago
    Google's OpenDNS servers are not down, they are recursive resolvers, and are not getting data from Dyn's authoritative resolvers. OpenDNS caches DNS data for longer than other recursive DNS providers (they call it smartcache, IIRC), which is why they are still working.
  • deathanatos 10y ago
    Google's DNS has been working all day here. The problem is that Dyn's DNS server is being DDoS'd; if you request a record that the authoritative DNS server for is hosted by Dyn, then when you query Google's DNS for that record, then Google's server needs to make a query to Dyn, which is down, and thus, your query fails. But queries to Google for non-Dyn domains will continue to work just fine. OpenDNS works because, as another poster notes, that, for better or worse, they don't strictly obey TTLs: https://news.ycombinator.com/item?id=12762429 https://news.ycombinator.com/item?id=12762429
  • bostonvaulter2 10y ago
    Here's the link that verifies OpenDNS's addresses (if you're wary of trusting a single HN comment): https://use.opendns.com/ https://use.opendns.com/
  • newsat13 10y ago
    Sorry for the confusion about saying Google NS is down :-) I meant dig heroku.com @8.8.8.8 does not work (as pointed by some other poster it is because google NS honors TTL but opendns does not)
• halayli 10y ago
  github.com seems to be down because of this.
• Kaedon 10y ago
  What other providers would you recommend than Dyn? Route53? Cloudflare? Something else?
• bgentry 10y ago
  If you're having issues with people accessing your running Heroku apps, it's likely because you're running your DNS through herokussl.com (with their SSL endpoint product) which is hosted on Dyn. If you can update your DNS to CNAME directly to the ELB behind it, it should at least make your site accessible.
  • asdf333 10y ago
    thanks for the tip! how did you determine the ELB address behind the ssl endpoint? edit: figured it out. What i did was do: nslookup your-SSL-endpoint.herokussl.com then you'll see the elb address. Switch to the openDNS servers helpfully pointed out by someone above first...
    • Artemis2 10y ago
      Presumably with something like `dig @208.67.220.220 -t CNAME <your site>.herokussl.com`. This uses the OpenDNS nameservers, that people have been reporting as working. Haven't tested it as I am on the go.
      • asdf333 10y ago
        thanks! figured it out but appreciate the help!!
        • natashabaker 10y ago
          I'm seeing "connection timed out; no servers could be reached". Anyone else seeing that when trying to run the above command?
          • asdf333 10y ago
            did you switch your computer's dns servers to openDNS? 208.67.222.222 208.67.220.220 (or specify dns server in the command)
• jrochkind1 10y ago
  Is it time for everyone to actually start using secondary name servers/DNS resolvers too from a different provider from primary? DNS _is_ built for this, for the very purpose of handling failure of the primary resolver, isn't it? Just most people don't seem to do it -- including major players? Or would that not actually solve this particular scenario?
  • sinap 10y ago
    The attack is on the authoritative name servers, not a DNS resolver. A public DNS resolver will query the authoritative name server for a record if it doesn't exist in it's cache.
    • mgamble 10y ago
      Agreed, but there is nothing stopping you from having the authoritative name servers for a domain with different providers. As someone previously said, DNS was designed for this.
      • Symbiote 10y ago
        It's used to be common for universities to do this, mine still does: ic.ac.uk. 45665 IN NS ns1.ic.ac.uk. ic.ac.uk. 45665 IN NS ns2.ic.ac.uk. ic.ac.uk. 45665 IN NS ns0.ic.ac.uk. ic.ac.uk. 45665 IN NS authdns1.csx.cam.ac.uk. (and Cambridge use Imperial College as a secondary) but the best-known American universities are on cloud providers now.
    • jrochkind1 10y ago
      Can you have secondary name servers too though? And would it have worked to avoid outage for domains doing such in this case?
  • danyork 10y ago
    Yes, I think this attack has brought to everyone's attention that many companies have gone away from what used to be the extremely common practice of having your authoritative DNS serving shared across multiple DNS hosting providers. This would have addressed the issue... and we're seeing that by the end of the day many of these sites have gone to having multiple providers.
• paulddraper 10y ago
  I thought DNS (particularly public) was basically immune to DDoS? If one DNS server is down, use the cached result or another server. DNS is some of the most distributable, cachable data I can imagine.
  • snug 10y ago
    Depends on how many PoPs they have. Looks like they have 4 easter US.[0] If they are seeing large attacks that Krebs saw a few weeks ago, that could certainly be enough to take down one or two, and then causing redirected traffic to take down the other two. I used to work for a DNS/DDoS provider, and this was a very real problem. Leave the PoPs that are being affected out, or risk overloading the other PoPs by overloading real traffic. Before moving the other traffic, you also have to worry about blocking the DDoS traffic otherwise you're just redirecting them to the other PoPs. Mitigating DDoS attacks are not fun, and hard to block. [0]http://dyn.com/dns/network-map/ http://dyn.com/dns/network-map/
  • acdha 10y ago
    Some sites intentionally disable that, however, by setting a short TTL on replies. The idea is usually that it allows them to very quickly adjust to hardware failures or load across datacenters but it has the consequence of making your infrastructure comparatively brittle.
• octoploid 10y ago
  It is spreading to other DNS providers, too: https://status.fastly.com/ https://status.fastly.com/ www.ft.com is unreachable for example.
  • snug 10y ago
    Fastly is simply putting up a status page so they aren't contacted about issues, and letting them know it's about DYN. And they are having internal issues with communications like zendesk.
• deleted 10y ago
  [deleted]
• Kluny 10y ago
  Highrise seems to be having problems, as seen by email errors when we forward email to Highrise dropboxes.
• elwell 10y ago
  To get on github you can add to your /etc/hosts: 192.30.253.113 github.com 151.101.32.133 assets-cdn.github.com And it seems faster than normal right (less users). Edit; for profile pics include: 151.101.32.133 avatars0.githubusercontent.com 151.101.32.133 avatars1.githubusercontent.com 151.101.32.133 avatars2.githubusercontent.com 151.101.32.133 avatars3.githubusercontent.com 151.101.32.133 avatars4.githubusercontent.com 151.101.32.133 avatars5.githubusercontent.com
  • xyclos 10y ago
    how about npm?
    • puddintane 10y ago
      I was able to access everything by changing DNS as mentioned in the other posts [1]. [1] https://news.ycombinator.com/item?id=12762841 https://news.ycombinator.com/item?id=12762841 edit Of course this is if your local policy allows you to change this!
    • elwell 10y ago
      you can get the ip from a different location using this: https://www.whatsmydns.net/ https://www.whatsmydns.net/
  • MattConfluence 10y ago
    How about *.github.io? Edit: saw your other reply and looked it up myself, it's 23.235.33.133
    • elwell 10y ago
      I don't think /etc/hosts will work with wildcard subdomains.
• edcastano 10y ago
  The great irony: http://www.isitdownrightnow.com/yesware.com.html http://www.isitdownrightnow.com/yesware.com.html
• dEnigma 10y ago
  I was not aware of the attacks going on until this happened: 1. Tried to download "Unknown Horizons" (game featured recently on Hacker News) binary, github-link doesn't work. 2. Think "Ok, might be an old link", google their github-repository, github appears down. 3. Try accessing github status website, is down. 4. Interested, try to visit github status twitter account, twitter is down. Really weird experience, normally at least the second source of news on a downed website I try during an attack works.
  • jeppebemad 10y ago
    Had a similar experience. When I went to confirm on twitter, that was down too. I was able to acces Twitter from my phone though, where I found a ton of tweets saying "Twitter down!". Strange.
    • alexmorenodev 10y ago
      Funny how we're able to tell that twitter's down on twitter, but here in Brazil, when whatsapp was down, nobody could use their own whatsapp to ask or tell if whatsapp was down.
• ifelsehow 10y ago
  Reposting imglorp's comment on the root of the comment tree, as it's buried currently. This should restore service for those desperately needing to access Github etc ;) > ....point your machine or router's DNS to use opendns resolvers instead of your regular ones: 208.67.222.222 and 208.67.220.220
• x2398dh1 10y ago
  Currently I am able to get into every site on the web, including GitHub, by using a VPN service based in Hong Kong.
• Artemis2 10y ago
  PayPal, Braintree, Spreedly down. Some companies are going to lose money today...
• tedmiston 10y ago
  Anyone else spend the morning thinking the problem was their setup? I've been flushing my system DNS cache, Chrome's DNS cache, changing DNS servers, rebooting my router, turning VPN on/off, etc.
  • mudge 10y ago
    Yep, I did the same.
  • stuntmachine 10y ago
    Yeah. :/ It happened to be at the same time I was getting things configured to connect to a new VPN that I hadn't used before for the first time. Until about 7am today my home network was a 10.0.0.0/8 network. VPN kept bombing in the last phase of connecting and I couldn't figure out why, so I thought it was an IP conflict with my internal network range. So naturally, I then went into my router and changed my subnet for my entire home network to the more common 192.168.1.0/24 range to see if it'd help. It didn't. Until suddenly VPN "just worked" -- which makes me wonder if I needed to change my network at all to begin with. Then I started experiencing all sorts of weird issues where the Internet seemed to disappear from one minute until the next. Then I hit IRC when things finally stabilized and see "Did you hear about Dyn?". My reaction: wut. TL;DR: I rearchitected my home network at 7am for no reason.
• arp 10y ago
  Here's how to add static mappings temporarily to survive through the outage: https://www.reddit.com/r/sysadmin/comments/58o5mp/dyn_dns_ddos_pt_2/d923yvw/ https://www.reddit.com/r/sysadmin/comments/58o5mp/dyn_dns_dd...
• Animats 10y ago
  "digikey.com", the big electronic part distributor, is currently inaccessible. DNS lookups are failing with SERVFAIL. Even the Google DNS server (8.8.8.8) can't resolve that domain. Their DNS servers are "ns1.p10.dynect.net" through "ns4.p10.dynect.net", so it's a Dyn problem. This will cause supply-chain disruption for manufacturers using DigiKey for just-in-time supply. (justdownforme.com says the site is down, but downforeveryoneorjustme.com says it's up. They're probably caching DNS locally.)
• invisiblep 10y ago
  Why not use: OpenDNS - recursive DNS Cloudflare (DNS only) - authoritative DNS Both services are free and distributed across the world.
  • q3k 10y ago
    Dyn was supposed to be distributed too. The takeaway here should be to use multiple authoritative DNS providers, not a single (even if better) one.
• foxhop 10y ago
  So I had hardcoded my DNS server to googles, aka: dig @8.8.4.4 github.com +short I was not getting an answer. However using my routers/dhcp/ISP to set my DNS server, I am able to get answers: dig github.com +short 192.30.253.112
  • snug 10y ago
    Cached locally? dig +trace github.com
• brooklyndude 10y ago
  Why does it always have to be a "Nation State", have been hanging out with 17 year old's that knew far more about DNS configs than a room of "Cyber-Security-Professisonals", they were clueless, these kids could run circles around them. Kids.
• tbarbugli 10y ago
  I am very surprised this is not getting that much attention on national news.
• fatherzeus 10y ago
  For people in need of the IPs for their respective services. You can find them here: ipaddress.com or any of the other similar services
• tim_armandpour 10y ago
  I wanted to provide an update on the PagerDuty service. At this time we have been able to restore the service by migrating to our secondary DNS provider. If you are still experiencing issues reaching any pagerduty.com addresses, please flush your DNS cache. This should restore your access to the service. We are actively monitoring our service and are working to resolve any outstanding issues. We sincerely apologize for the inconvenience and thank our customers for their support and patience. Real-time updates on all incidents can be found on our status page and on Twitter at @pagerdutyops and @pagerduty. In case of outages with our regular communications channels, we will update you via email directly. In addition you can reach out to our customer support team at support@pagerduty.com or +1 (844) 700-3889. Tim Armandpour, SVP of Product Development, PagerDuty
  • pjlegato 10y ago
    "Challenges" is exactly the sort of Dilbertesque euphemism that you should never say in a situation like this. Calling it a "challenge" implies that there is some difficult, but possible, action that the customer could take to resolve the issue. Since that is not the case, this means either you don't understand what's going on, or you're subtly mocking your customers inadvertently. Try less to make things sound nice and MBAish, and try more to just communicate honestly and directly using simple language.
  • kirizt 10y ago
    I appreciate the update, but your service has been unavailable for hours already. This is unacceptable for a service whose core value is to ensure that we know about any incidents.
    • warmwaffles 10y ago
      You apparently don't understand DNS. DynDNS is a very reputable DNS provider that twitter and many others use. For someone to take them down requires a massive amount of effort. Running a redundant DNS provider is expensive as all hell.
      • deleted 10y ago
• Rapzid 10y ago
  Looks like github and braintree both got AWS dns servers mixed in about the same time. Did they both switch over or is Dyn working with AWS on this?
• Animats 10y ago
  Github is currently inaccessible. Can you still compile Rust programs that depend on Github files?
  • clay_to_n 10y ago
    If you need to, you can edit your hosts file to include github's IP addresses (mentioned elsewhere in this thread): 192.30.253.113 github.com 151.101.4.133 assets-cdn.github.com
  • steveklabnik 10y ago
    First of all, the only thing that'd matter is for modifying your dependencies at the moment. If you've previously built the project, and don't touch your deps, GitHub won't be hit. Second, Cargo only depends on GitHub for the index. For more: http://integer32.com/2016/10/08/bare-minimum-crates-io-mirror-plus-one.html http://integer32.com/2016/10/08/bare-minimum-crates-io-mirro... That includes a link to a mirror run by integer32.
• kakarot 10y ago
  Don't be a dick. I'm sure their staff has a giant collective migraine right now.
• blobbob2 10y ago
  This is a false flag operation by the Obama administration so they can blame it on Russia. If you notice all of the rhetoric from the democrats blaming Russia with no evidence. Then you understand that Russia has nothing to gain from such an attack, it is crystal clear.
• Mizza 10y ago
  Although I don't like to to recommend Google products, they provide a provide a public DNS-over-HTTPS interface that should be useful for people who want to add specific entries into their /etc/hosts files: https://dns.google.com/query?name=github.com&type=A&dnssec=true https://dns.google.com/query?name=github.com&type=A&dnssec=t...
• djhworld 10y ago
  At work earlier we was seeing hostname resolution errors with applications trying to contact amazon s3 from on premises infrastructure. This was in eu-west-1, but it coincided with a bunch of other systems in the organisation having problems at the same time. Additionally CloudWatch logs seemed to be completely broken for about 30 minutes on the Amazon Console.
• Animats 10y ago
  So who was prepared for this? Pornhub: pornhub.com: Name Server: ns1.p44.dynect.net Name Server: ns2.p44.dynect.net Name Server: ns3.p44.dynect.net Name Server: ns4.p44.dynect.net Name Server: sdns3.ultradns.biz Name Server: sdns3.ultradns.com Name Server: sdns3.ultradns.net Name Server: sdns3.ultradns.org ultradns.biz: Name Server: PDNS196.ULTRADNS.ORG Name Server: ARI.ALPHA.ARIDNS.NET.AU Name Server: ARI.BETA.ARIDNS.NET.AU Name Server: ARI.GAMMA.ARIDNS.NET.AU Name Server: ARI.DELTA.ARIDNS.NET.AU Name Server: PDNS196.ULTRADNS.NET Name Server: PDNS196.ULTRADNS.COM Name Server: PDNS196.ULTRADNS.BIZ Name Server: PDNS196.ULTRADNS.INFO Name Server: PDNS196.ULTRADNS.CO.UK
  • Animats 10y ago
    Looks like Pagerduty just dumped Dyn: pagerduty.com: Name Server: NS-219.AWSDNS-27.COM Name Server: NS-1198.AWSDNS-21.ORG Name Server: NS-1569.AWSDNS-04.CO.UK Name Server: NS-739.AWSDNS-28.NET Pagerduty annoucement: "If you are having issues reaching any pagerduty.com address please flush your DNS cache to resolve the issue."
  • Animats 10y ago
    Github just added AWS DNS: github.com: Name Server: ns2.p16.dynect.net Name Server: ns-1283.awsdns-32.org. Name Server: ns-1707.awsdns-21.co.uk. Name Server: ns-421.awsdns-52.com. Name Server: ns1.p16.dynect.net Name Server: ns4.p16.dynect.net Name Server: ns3.p16.dynect.net Name Server: ns-520.awsdns-01.net.
  • razster 10y ago
    ultradns.biz has been down as well.
    • Animats 10y ago
      "ultradns.biz" is not responding to pings, but their DNS servers are responding to DNS queries properly: nslookup > server pdns196.ultradns.biz Default server: pdns196.ultradns.biz Address: 156.154.66.196#53 Default server: pdns196.ultradns.biz Address: 2610:a1:1015::e8#53 > pornhub.com Server: pdns196.ultradns.biz Address: 156.154.66.196#53 Name: pornhub.com Address: 31.192.120.36 Right now, if your site is in trouble, I'd suggest getting UltraDNS service and AWS DNS service, and some obscure service as well, and put them them all in your domain registration. DNS service is cheap. Get some redundancy going. We have no idea how long this DDOS attack will last. It's not costing the attackers anything. They might leave it running for days.
• elmigranto 10y ago
  I've managed to (seemingly) save my browsing with Yandex DNS: 77.88.8.8 77.88.8.1 https://dns.yandex.ru https://dns.yandex.ru
  • elwell 10y ago
    I'm sure yandex is safe, but I'm wary of using a anything dns.*.ru to route my traffic to potential phishing versions of sites.
    • elmigranto 10y ago
      If it makes you more at ease, use this one :) https://dns.yandex.com https://dns.yandex.com
• BoycieRules 10y ago
  unsubscribe
• dev_1024 10y ago
  How come you can access these sites from some countries? I imagine there are lots of name servers and that the attackers are specifically targeting servers for US?
• deleted 10y ago
  [deleted]
• azaydak 10y ago
  Quick question for you all. Just two days ago I registered two domain names at dynu (not dyn). Early this morning I a cold call from a company in India who knew the domain names and my phone number and was calling to ask if I wanted them to help me manage my website cheaply. Also, this morning I got a spam text from someone who claimed to by godaddy offering the same thing. Now I protect my number really well so this is the first time in 5+ years that I ever got spam texts or calls to my number. Do you think Dynu was also hacked?! Or maybe Dynu sells client numbers (which is how the guy in India claimed to get my number) and it was just by random chance that this happened at the same time as the Dyn hack.
  • shortstuffsushi 10y ago
    Fwiw, this isn't a hack, this is a DDoS (denial of service). It seems almost certain that your information was either given out by dynu, or your WHOIS record isn't protected. Check your domains out with your favorite WHOIS tool first. Otherwise... time for an awkward conversation with dynu.
    • azaydak 10y ago
      Right, hack wasn't the right word. Anyway, thanks for the info.
  • danyork 10y ago
    Agree with shortstuffsushi that this is just someone getting your domain name info and spamming you. It sadly happens all the time. Go to http://whois.icann.org/en http://whois.icann.org/en and enter your domain name and see what info is public about you. If all your info is public, you may want to see if your registrar offers "private" registration where your info does not appear in WHOIS.
• peatmoss22 10y ago
  Need to get in to dyn.com to download your zone files add this to your hosts file: 204.13.248.106 www.dyn.com 204.13.248.106 dyn.com 216.146.41.66 manage.dynect.net 151.101.33.7 static.dyn.com
• peatmoss22 10y ago
  Need to get in to dyn.com to download your zone files add this to your hosts file: 204.13.248.106 www.dyn.com 204.13.248.106 dyn.com 216.146.41.66 manage.dynect.net 151.101.33.7 static.dyn.com
• DenisM 10y ago
  The DDoS problems, at least those not related to spoofing IPs, could be curtailed if we provide a strong incentive to the ISPs to work on it. Let's hold the ISPs financially liable for the harmful traffic that comes from their network. If a client reports a harmful IP to the ISP, every bit of subsequent traffic sent from that IP to this client carries a penalty. Yeah, I know, routing tables are small, yada yada. If we put thumbscrews to the ISPs they will find a way to block a few thousands IPs of the typical botnet, even it requires buying new switches from Cisco & co. Incentives drive behavior.
  • Florin_Andrei 10y ago
    Put the thumbscrews on the IoT manufacturers instead, so they don't release widgets with bad security, so the problem is eliminated at its root. You wouldn't allow car manufacturers to sell cars with faulty airbags, why do we allow device manufacturers to provide plentiful firepower for bad actors?
    • DenisM 10y ago
      With ISPs it's a lot easier - you know who your ISP is, so either they respect your blacklist or they automatically owe you money. How would you even start chasing a manufacturer of a cheap IP cam from China? How many of them can you chase at once?
• mmaunder 10y ago
  Third attack underway: https://twitter.com/AlexJamesFitz https://twitter.com/AlexJamesFitz (as of 10 mins ago)
• lips 10y ago
  How many DNS services ala Dyn exist? Is it not still massively significant that a successful attack can be launched on even one of these?
• dmalvarado 10y ago
  This may be dumb, but someone enlighten me: If this kind of attacking does escalate, wouldn't it be possible to simply cut off requests from outside the United States at the points of entry? Basically, turning the US into an intranet?
  • danyork 10y ago
    We don't know yet, but the attack very easily could have been coming from a botnet of devices entirely inside the US. Geographic borders don't matter much at all for the Internet.
    • dmalvarado 10y ago
      But even if it were, the creator of the botnet would first have to gain control and then issue a command, right? How would either of those things be possible from outside if there was no connection into the US?
      • danyork 10y ago
        Well, there's pretty much no way to impose the geographic borders of the US onto the Internet. Our networks here in the US are all global and integrated with other networks all around the world. The only places where this kind of geographic control is possible are countries like Iraq, Iran, China and others where the government controls all the ISPs. Countries with more freedom have a free flow of information and packets - and to me that is a very GOOD thing.
• shortstuffsushi 10y ago
  In (well, after) attacks like this, and really any other massive DDOS, shouldn't it be possible to identify potential botnets and try to take them out (notify their owners that they're being used, notify their hosting providers, etc) so that they can't be used again in the future?
• LeanderK 10y ago
  Can someone explain why this is so bad? I think the internet handled the downtime of Dyn pretty great, not reaching github wasn't exactly pleasing, but i added the ip temporary to /etc/hosts and the problem was solved. Isn't the best strategy to accept that attacks will continue and systems may go down and design for resilience? If so this attack can serve as a warning and as a check that we can handle these types of attacks. I am a bit exaggerating, but i would imagine that constant attacks keep the internet resilient and healthy. An unchallenged internet may be the greater risk.
  • rak00n 10y ago
    You're assuming we'll build immunity fast enough. What if we don't? Attacks at this scale can bring a significant part of Internet down. The economic affect can be just as bad as a war.
    • LeanderK 10y ago
      but just pretending no one will try bad things won't help either. It may speed up the progress and will prevent bringing a significant part of the internet down in the future and also remind everybody that these things can (and probably will) happen.
• edgartaor 10y ago
  I'm curious. What kind of infrastructure you need to make this massive attack?
  • saidajigumi 10y ago
    via [1]: "Dyn says today's DDoS are in part being caused by Mirai botnet, which recently caused record-sized attack [2]" [1] https://twitter.com/AlexJamesFitz/status/789562789920636928 https://twitter.com/AlexJamesFitz/status/789562789920636928 [2] https://krebsonsecurity.com/2016/10/source-code-for-iot-botnet-mirai-released/ https://krebsonsecurity.com/2016/10/source-code-for-iot-botn... tl;dr: a substantial part of this attack is a botnet of IoT devices.
• Animats 10y ago
  Analysis of the Mirai botnet: [1] This is worth reading. It has links to copies of the code and names the known control servers. Quite a bit is known now about how this thing works. The bots talk to control servers and report servers. The attacker appears to communicate with the report servers over Tor. [1] http://blog.level3.com/security/grinch-stole-iot/ http://blog.level3.com/security/grinch-stole-iot/
• rybosome 10y ago
  I'm wondering, from a regulatory perspective, what might be done to mitigate DDoS attacks in the future? From comments made on this and other similar posts in the past, I've gathered the following: 1) Malicious traffic often uses a spoofed IP address, which is detectable by ISPs. What if ISPs were not allowed to forward such traffic? 2) There is no way for a service to exert back pressure. What if there was? e.g. send a response indicating the request was malicious (or simply unwanted due to current traffic levels), and a router along the way would refuse to send follow up requests for some time. There is HTTP status code 429, but that is entirely dependent on a well-behaved client. I'm talking about something at the packet level, enforced by every hop along the way. 3) I believe it is suspected that a substantial portion of the traffic is from compromised IoT devices. What if IoT devices were required to continually pass some sort of a health check to make other HTTP requests? This could be enforced at the hardware/firmware level (much harder to change with malware), and, say, send a signature of the currently running binary (or binaries) to a remote server which gave the thumbs up/down.
  • tim333 10y ago
    One thing that occured to me regulations wise is to require IoT devices to have some minimum level of security such as a unique hard password rather than it just being "admin" or some such. You could enforce it for items sold in the US or EU and the Chinese manufacturers would probably follow so their goods could be sold easily.
    • nkristoffersen 10y ago
      I've noticed a lot of wifi routers are doing this now. Which is pretty great. All appear to be unique passwords for each router.
  • thomask0 10y ago
    Let ISP's shutdown traffic of customers with compromised devices.
    • nkristoffersen 10y ago
• wweiss1230 10y ago
  How can I, a proficient web developer but one with little experience working directly with its underlying infrastructure, help in whatever effort is being down to thwart this and related attacks? I feel a moral obligation to help as these attacks seem a grave threat to our economy and could cause unrest given the current political climate. Thanks.
  • dmourati 10y ago
    Read all the analysis you can to form a better understanding of how this all works. Use that information to design and run more resilient services in the future. Teach what you have learned to others.
• pawal 10y ago
  DNS was designed so that you can have multiple operators for your authoritative name servers. Who would have thought adding a spof to your infrastructure would ever be a problem?
• danyork 10y ago
  Journalist and security researcher Brian Krebs believes this is someone doing a DDoS as payback for research into questionable "DDoS mitigation services" that he and Dyn's Doug Madory did. Doug just presented his results yesterday at NANOG and Krebs believes this is payback. Read more: https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twit...
  • LordHumungous 10y ago
    If so that's a quick turnaround.
    • danyork 10y ago
      Well, Krebs sees this as an extension of the attacks that took down his site a few weeks ago after he wrote about this research. So he wrote about it, attackers take down his site. His co-author Doug Madory speaks about it, attackers take down Madory's employer's site. Krebs indicates in an update at the end that a source had heard rumors in criminal channels that an attack against Dyn was being planned. Doug Madory's presentation was on the agenda for NANOG and so attackers would have had plenty of time to know about it.
• deleted 10y ago
  [deleted]
• deleted 10y ago
  [deleted]
• danyork 10y ago
  Other HN threads on related articles: Krebs on Security: https://news.ycombinator.com/item?id=12761859 https://news.ycombinator.com/item?id=12761859 NY Times: https://news.ycombinator.com/item?id=12765652 https://news.ycombinator.com/item?id=12765652 Bloomberg: https://news.ycombinator.com/item?id=12763501 https://news.ycombinator.com/item?id=12763501 Dyn: https://news.ycombinator.com/item?id=12764650 https://news.ycombinator.com/item?id=12764650
• danyork 10y ago
  There's a bit of exquisite irony in the fact that just yesterday an article on the Dyn blog was: Recent IoT-based Attacks: What Is the Impact On Managed DNS Operators? - http://hub.dyn.com/traffic-management/recent-iot-based-attacks-what-is-the-impact-on-managed-dns-operators http://hub.dyn.com/traffic-management/recent-iot-based-attac... It's a good piece about how IoT-based DDoS attacks are carried out. And now Dyn has the answer... HN thread about that article at: https://news.ycombinator.com/item?id=12764650 https://news.ycombinator.com/item?id=12764650
• anonymousjunior 10y ago
  No idea if this would work, but could people theoretically just ping flood the IOT devices involved to mitigate the attack? They run some sort of web server since most devices provide some web interface, so clearly there's a port open which could be hit if the IP is know, and with the shoddy security in these devices I'd wonder if their local (likely low performance) hardware would be susceptible to something as simple as a ping flood attack.
• wav-part 10y ago
  Why is there even a concept of managed DNS ? Arnt we already paying >$1M/yr so that we can get 32 bit integer from a string ? This does not make sense.
• dboreham 10y ago
  What this event shows is that using DNS as a load routing/balancing mechanism is a bad idea (because that's why folks have low TTL and an inability to specify truly redundant secondaries).