7 ms·

Are we going to start a betting pool as to why? I put $20 on DDoS, but I'm hedging $5 that someone spilled their drink on something important and quickly got o

Are we going to start a betting pool as to why? I put $20 on DDoS, but I'm hedging $5 that someone spilled their drink on something important and quickly got out of there before anyone noticed.

epx 8y ago
The janitor needed that socket to plug the vacuum
- 706f6f70 8y ago
  Experimental Roomba built up a static charge and fried something important.
Escapado 8y ago
Wouldn't DDoS be incredibly hard to pull off given YouTubes vast resources? I mean they generate so much traffic all the time and serve so many requests a second I don't know if botnets of that size exist. Anyone here more knowledgeable?
- corodra 8y ago
  Would it involve hitting whoever/whatever handles their DNS like that one attack about a year ago? FYI: I legit am retarded to such things and my networking knowledge is...meh. So please correct me.
  - orangepenguin 8y ago
    DNS is resolving because people can get to the site. Because of the distributed nature of YouTube's resources (at DNS as well as other layers), it's very unlikely to be DDoS related. Static content loads, but video content doesn't. Seems like an application error, or a problem with media streaming infrastructure.
- deusofnull 8y ago
  Yeah i feel like a DDoS that big would be afecting other google services right now...
- mark-r 8y ago
  Maybe it's a brain teaser - if you can answer how to DDoS Youtube, they'll call you up for an interview.
  - justtopost 8y ago
    I could tell you a few ways, but would never work for them out of principle. They can find their own bugs.
- franciskim 8y ago
  I believe you can still bring it down through a distributed amplification attack. Refer to Google for more info.
- Obi_Juan_Kenobi 8y ago
  I suppose if you had something new and incredibly powerful, this would be a great way to show it off. Still more likely that YT just made an oopsie.
- srcmap 8y ago
  A few hundred AI learning instances were started to train on youtube videos to "understand" humanity. They have become self aware and started to consume double amount of CPU/GPU/Disk/RAM/Network resource every few seconds - a skynet/super nova moment.
jarsin 8y ago
NSA was installing a wire tap and sliced some fiber.
- LeoPanthera 8y ago
  That would not cause a global outage. YouTube has caches in all the major ISPs.
  - deusofnull 8y ago
    maybe nsa was installing a whole bunch of wire taps?
    - corodra 8y ago
      With the help of aliens since they have a better vantage point of where all the cables are... because, you know, the Earth is fla-... nope. Can't do it. I can't go that far. I can only say "so much" stupid. That's too far.
__george__ 8y ago
I bet it’s a core network issue since it went down globally at the same time.. they deploy new binaries slowly so that shouldn’t cause it. Maybe some config was changed without testing?
- corodra 8y ago
  A. You're not being fun. 2. Good point. III. One thing I did notice. I had some other videos in other tabs. They still loaded through. I could skip around and it streamed in video like nothing was wrong. When I refresh, bust. Quatre: Five......
- kenrose 8y ago
  Agreed. I'd suspect it'll be something similar to Google's 2016 GCP outage: https://status.cloud.google.com/incident/compute/16007 https://status.cloud.google.com/incident/compute/16007 (I'm thinking something like router firmware or BGP).
- lavaslice 8y ago
  i think that if this is the case, it's an fast and easy fix, but they haven't solved yet so i think this is not the issue
- djsumdog 8y ago
  Is it down globally? I've has some EU people on Mastodon say it's still up for them.
sandov 8y ago
I'm betting it has to do with video indexation. I was watching a long video just fine, then I searched for something else and clicked on one result. Error message.
AceJohnny2 8y ago
> I put $20 on DDoS I'd take you on that, except I have no anonymized payment account. Otherwise, I'd bet $100 that it's not. Only Google has the scale to DDoS Google.
- corodra 8y ago
  Hey! Conspiracy time! Maybe it's an inner splinter rogue group within Google that's been secretly hating on a Youtube team and decided to get the ultimate payback for like... not inviting them to a lan party...or something...aliens ate all the cheesey poofs.
  - ddtaylor 8y ago
    It's crab people for sure.
    - mindcrime 8y ago
      I'm not saying it's aliens, but...
      - djsumdog 8y ago
        It's the sunspot thing. They took down the Hubble, that other telescope, that random lab, and now the frog people Alex Jones warned us about are coming to inject you with the gay and ship you on a raft made out of Michael Moore to Cuba.
        mindcrime 8y ago
        It's the sunspot thing. They took down the Hubble, that other telescope, that random lab, and now the frog people Alex Jones warned us about are coming to inject you with the gay and ship you on a raft made out of Michael Moore to Cuba. All to conceal the real truth about what happened on 9/11 and how it connects to the plot to obfuscate what really happened at Pearl Harbor and how the Australian mafia actually had JFK assassinated to cover up the truth about Marilyn Monroe, Tupac, and Charles Proteus Steinmetz. A clever scheme, no doubt orchestrated by the evil consortium of George Westinghouse, Thomas Edison and Guglielmo Marconi. Hopefully Elon Musk, Robert Downey Jr. and Nikola Tesla will be able to bring the truth to light!
deleted 8y ago
[deleted]
burger_moon 8y ago
Fat fingered a script would be my guess. Not the first time that caused a 'big4' site to go down.
shiado 8y ago
DNS, it is always DNS.
trevyn 8y ago
My vote: Bad configuration got pushed.
- SteveNuts 8y ago
  I thought they had all sorts of crazy canary deployment processes that will automatically stop/rollback the deployment if the failure metrics start increasing?
  - corodra 8y ago
    Ran out of canaries? Well, could be an unforeseeable-wack-crazy-bug. Can't plan for every possible failure.
    - SteveNuts 8y ago
      That's very true, there's always a bug somewhere lurking, even if you're Google.
  - p0rkbelly 8y ago
    There was a GCP Global network outage maybe a year or two ago, where their canary of a canary (Yo dawg, I heard you like bugs, so I I put a but inside of your bug) caused a cascading network failure. It was a bad config push -- brownouts fighting with brown outs. Consistency issues. I bet this is happening.
nodesocket 8y ago
I'll take your $20 bet and raise you to $100. Not a DDoS.
khazhou 8y ago
$20 says it was Greg. Again!
aviv 8y ago
My bet is rm -f instead of cp -R.
analog31 8y ago
Somebody probably accidentally powered down the Web: https://home.cern/about/updates/2012/12/month-1991-web-spreads-beyond-cern https://home.cern/about/updates/2012/12/month-1991-web-sprea...
deleted 8y ago
[deleted]
richardknop 8y ago
My bet is data loss. Some engineer deleted MySQL table accidentally and they found out their backups are not working.
blihp 8y ago
Joe, the only YouTube support person in the office this evening, accidentally hit the pause button on the YouTube service in his admin console before stepping out for a long bathroom break. Hey, it could happen...
samstave 8y ago
The Saudis dont like all the attention?
hindsightRegret 8y ago
Metadata querying I imagine. The search was working, buffered videos and main site were loading, just not displaying videos. (so not network)
PuffinBlue 8y ago
I bet on Google Global Cache problem. YouTube was down for me through my residential ISP (which no doubt has internal caching via GGC) but through VPN was fine.

Are we going to start a betting pool as to why? I put $20 on DDoS, but I'm hedging $5 that someone spilled their drink on something important and quickly got out of there before anyone noticed.

epx 8y ago

The janitor needed that socket to plug the vacuum

706f6f70 8y ago
Experimental Roomba built up a static charge and fried something important.

Escapado 8y ago

Wouldn't DDoS be incredibly hard to pull off given YouTubes vast resources? I mean they generate so much traffic all the time and serve so many requests a second I don't know if botnets of that size exist. Anyone here more knowledgeable?

corodra 8y ago
Would it involve hitting whoever/whatever handles their DNS like that one attack about a year ago? FYI: I legit am retarded to such things and my networking knowledge is...meh. So please correct me.
- orangepenguin 8y ago
  DNS is resolving because people can get to the site. Because of the distributed nature of YouTube's resources (at DNS as well as other layers), it's very unlikely to be DDoS related. Static content loads, but video content doesn't. Seems like an application error, or a problem with media streaming infrastructure.
deusofnull 8y ago
Yeah i feel like a DDoS that big would be afecting other google services right now...
mark-r 8y ago
Maybe it's a brain teaser - if you can answer how to DDoS Youtube, they'll call you up for an interview.
- justtopost 8y ago
  I could tell you a few ways, but would never work for them out of principle. They can find their own bugs.
franciskim 8y ago
I believe you can still bring it down through a distributed amplification attack. Refer to Google for more info.
Obi_Juan_Kenobi 8y ago
I suppose if you had something new and incredibly powerful, this would be a great way to show it off. Still more likely that YT just made an oopsie.
srcmap 8y ago
A few hundred AI learning instances were started to train on youtube videos to "understand" humanity. They have become self aware and started to consume double amount of CPU/GPU/Disk/RAM/Network resource every few seconds - a skynet/super nova moment.

jarsin 8y ago

NSA was installing a wire tap and sliced some fiber.

LeoPanthera 8y ago
That would not cause a global outage. YouTube has caches in all the major ISPs.
- deusofnull 8y ago
  maybe nsa was installing a whole bunch of wire taps?
  - corodra 8y ago
    With the help of aliens since they have a better vantage point of where all the cables are... because, you know, the Earth is fla-... nope. Can't do it. I can't go that far. I can only say "so much" stupid. That's too far.

__george__ 8y ago

I bet it’s a core network issue since it went down globally at the same time.. they deploy new binaries slowly so that shouldn’t cause it. Maybe some config was changed without testing?

corodra 8y ago
A. You're not being fun. 2. Good point. III. One thing I did notice. I had some other videos in other tabs. They still loaded through. I could skip around and it streamed in video like nothing was wrong. When I refresh, bust. Quatre: Five......
kenrose 8y ago
Agreed. I'd suspect it'll be something similar to Google's 2016 GCP outage: https://status.cloud.google.com/incident/compute/16007 https://status.cloud.google.com/incident/compute/16007 (I'm thinking something like router firmware or BGP).
lavaslice 8y ago
i think that if this is the case, it's an fast and easy fix, but they haven't solved yet so i think this is not the issue
djsumdog 8y ago
Is it down globally? I've has some EU people on Mastodon say it's still up for them.

sandov 8y ago

I'm betting it has to do with video indexation. I was watching a long video just fine, then I searched for something else and clicked on one result. Error message.

AceJohnny2 8y ago

> I put $20 on DDoS I'd take you on that, except I have no anonymized payment account. Otherwise, I'd bet $100 that it's not. Only Google has the scale to DDoS Google.

corodra 8y ago
Hey! Conspiracy time! Maybe it's an inner splinter rogue group within Google that's been secretly hating on a Youtube team and decided to get the ultimate payback for like... not inviting them to a lan party...or something...aliens ate all the cheesey poofs.
- ddtaylor 8y ago
  It's crab people for sure.
  - mindcrime 8y ago
    I'm not saying it's aliens, but...
    - djsumdog 8y ago
      It's the sunspot thing. They took down the Hubble, that other telescope, that random lab, and now the frog people Alex Jones warned us about are coming to inject you with the gay and ship you on a raft made out of Michael Moore to Cuba.
      - mindcrime 8y ago
        It's the sunspot thing. They took down the Hubble, that other telescope, that random lab, and now the frog people Alex Jones warned us about are coming to inject you with the gay and ship you on a raft made out of Michael Moore to Cuba. All to conceal the real truth about what happened on 9/11 and how it connects to the plot to obfuscate what really happened at Pearl Harbor and how the Australian mafia actually had JFK assassinated to cover up the truth about Marilyn Monroe, Tupac, and Charles Proteus Steinmetz. A clever scheme, no doubt orchestrated by the evil consortium of George Westinghouse, Thomas Edison and Guglielmo Marconi. Hopefully Elon Musk, Robert Downey Jr. and Nikola Tesla will be able to bring the truth to light!

deleted 8y ago

[deleted]

burger_moon 8y ago

Fat fingered a script would be my guess. Not the first time that caused a 'big4' site to go down.

shiado 8y ago

DNS, it is always DNS.

trevyn 8y ago

My vote: Bad configuration got pushed.

SteveNuts 8y ago
I thought they had all sorts of crazy canary deployment processes that will automatically stop/rollback the deployment if the failure metrics start increasing?
- corodra 8y ago
  Ran out of canaries? Well, could be an unforeseeable-wack-crazy-bug. Can't plan for every possible failure.
  - SteveNuts 8y ago
    That's very true, there's always a bug somewhere lurking, even if you're Google.
- p0rkbelly 8y ago
  There was a GCP Global network outage maybe a year or two ago, where their canary of a canary (Yo dawg, I heard you like bugs, so I I put a but inside of your bug) caused a cascading network failure. It was a bad config push -- brownouts fighting with brown outs. Consistency issues. I bet this is happening.

nodesocket 8y ago

I'll take your $20 bet and raise you to $100. Not a DDoS.

khazhou 8y ago

$20 says it was Greg. Again!

aviv 8y ago

My bet is rm -f instead of cp -R.

analog31 8y ago

Somebody probably accidentally powered down the Web: https://home.cern/about/updates/2012/12/month-1991-web-spreads-beyond-cern https://home.cern/about/updates/2012/12/month-1991-web-sprea...

deleted 8y ago

[deleted]

richardknop 8y ago

My bet is data loss. Some engineer deleted MySQL table accidentally and they found out their backups are not working.

blihp 8y ago

Joe, the only YouTube support person in the office this evening, accidentally hit the pause button on the YouTube service in his admin console before stepping out for a long bathroom break. Hey, it could happen...

samstave 8y ago

The Saudis dont like all the attention?

hindsightRegret 8y ago

Metadata querying I imagine. The search was working, buffered videos and main site were loading, just not displaying videos. (so not network)

PuffinBlue 8y ago

I bet on Google Global Cache problem. YouTube was down for me through my residential ISP (which no doubt has internal caching via GGC) but through VPN was fine.