5 ms·

Hetzner mail to customers: 1 megawatt more power due to leap second

82 points by Uchikoma 14y ago

(Was Google translate, now from the comment of imaginator below, thanks) Better translation of their message to English speaking customers: During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt. The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers. According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can... In order to reduce CPU load to a normal level again, a restart of the whole system is necessary in many cases. First, a soft reboot via the command line should be attempted. Failing that, you have the option of performing a hardware reset via the Robot administration interface. For this, select menu item "Server" and the "Reset" tab for the respective server in the administration interface. Please do not hesitate to contact us, should you have any queries. Kind regards, Hetzner Online AG Stuttgarter Str. 1 91710 Gunzenhausen / Germany info@hetzner.de http://www.hetzner.com

StavrosK 14y ago
No need for Google translate: During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt. The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers. According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can-freeze-1629805.html http://www.h-online.com/open/news/item/Leap-second-Linux-can... Please do not hesitate to contact us, should you have any queries.
imaginator 14y ago
Better translation of their message to English speaking customers: During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt. The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers. According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can-freeze-1629805.html http://www.h-online.com/open/news/item/Leap-second-Linux-can... In order to reduce CPU load to a normal level again, a restart of the whole system is necessary in many cases. First, a soft reboot via the command line should be attempted. Failing that, you have the option of performing a hardware reset via the Robot administration interface. For this, select menu item "Server" and the "Reset" tab for the respective server in the administration interface. Please do not hesitate to contact us, should you have any queries. Kind regards, Hetzner Online AG Stuttgarter Str. 1 91710 Gunzenhausen / Germany info@hetzner.de http://www.hetzner.com http://www.hetzner.com
- Uchikoma 14y ago
  Thanks.
- imaginator 14y ago
  just remember: date -s "`date`" solves it. No reboot, no stopping and starting NTPd.
  - hendi_ 14y ago
    date -s "$(LC_ALL=C date)" is the better option which also works on system with a locale != C
metabrew 14y ago
Here's the requisite graph http://i.imgur.com/hsUDE.png http://i.imgur.com/hsUDE.png
- dfc 14y ago
  where did you get that data?
  - atesti 14y ago
    Maybe from here: http://www.heise.de/newsticker/meldung/Schaltsekunden-Bug-in-Linux-verschwendet-Strom-1631325.html http://www.heise.de/newsticker/meldung/Schaltsekunden-Bug-in... Heise.de got it directly from Hetzner, they acknowledge and thank them for providing it in the update of the article
- iand 14y ago
  Look how spiky the power usage is normally. Is that all the cron jobs set to run at the top of the hour?
  - danielweber 14y ago
    Maybe people should get discounts for running their cron jobs at the 50-minute mark.
- cellularmitosis 14y ago
  /standard rant about the Y-axis not being anchored at zero, which will cause most people to be mislead. UPDATE: here's an illustration. compare this: http://i.imgur.com/f3m25.png http://i.imgur.com/f3m25.png to this: http://i.imgur.com/6z2cS.png http://i.imgur.com/6z2cS.png (those two graphs represent the same data)
  - Dylan16807 14y ago
    Really? Who is both going to understand leap seconds, and also look at that graph and think power consumption went up tenfold. I'd much rather see the detail that zoom gives.
etaty 14y ago
OVH https://twitter.com/olesovhcom/status/220142781895671810 https://twitter.com/olesovhcom/status/220142781895671810
- deleted 14y ago
  [deleted]
ck2 14y ago
If the leap-second does all this, just imagine the 32-bit rollover issue in 2038
- forgotusername 14y ago
  I'm imagining a whole bunch of bricked "classic" cars
  - daeken 14y ago
    I really can't imagine why that would ever happen. What car would ever use a clock for anything essential, instead of a timer? Maybe I'm overestimating the intelligence of car developers, but it seems like the obvious, simple solution.
    - forgotusername 14y ago
      Engine won't start, service due 300 years ago. I'd be willing to bet $10 there's at least a few floating around with real time clocks, no idea what they'd be used for though. If nothing else there's always in-car navigation and entertainment which are definitely candidates, although I suppose less likely to result in a brick.
      - mattdeboard 14y ago
        Sorry, what?
        forgotusername 14y ago
        I'm engaging in what is commonly known as using one's imagination to make educated guesses at the possible electronic devices within an average motor vehicle (you know, that thing with wheels on it) where a real time clock susceptible to wraparound or overflow might be found, thus resulting in potential malfunction, or in plainer words, "the car won't start, the tuner won't play music, the GPS won't get a signal, this thing is about as useful as a brick".
kodisha 14y ago
Huh? I received my mail in English :)
efutch 14y ago
Perhaps nitpicking, but this is because of a bug in the leap second implementation, not in the leap second per se.
- Uchikoma 14y ago
  I'd guess it happens by the concurrence of a lep second and a leap second implementation bug. The bug without leap seconds would also not lead to problems.
imaginator 14y ago
Nice graph from Hetzner showing the spike: http://imgur.com/a/ykoup http://imgur.com/a/ykoup

82 points by Uchikoma 14y ago

StavrosK 14y ago

No need for Google translate: During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt. The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers. According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can-freeze-1629805.html http://www.h-online.com/open/news/item/Leap-second-Linux-can... Please do not hesitate to contact us, should you have any queries.

imaginator 14y ago

Better translation of their message to English speaking customers: During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level of IT power usage by approximately one megawatt. The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers. According to heise.de, various Linux distributions are affected by this. Further information can be found at: http://www.h-online.com/open/news/item/Leap-second-Linux-can-freeze-1629805.html http://www.h-online.com/open/news/item/Leap-second-Linux-can... In order to reduce CPU load to a normal level again, a restart of the whole system is necessary in many cases. First, a soft reboot via the command line should be attempted. Failing that, you have the option of performing a hardware reset via the Robot administration interface. For this, select menu item "Server" and the "Reset" tab for the respective server in the administration interface. Please do not hesitate to contact us, should you have any queries. Kind regards, Hetzner Online AG Stuttgarter Str. 1 91710 Gunzenhausen / Germany info@hetzner.de http://www.hetzner.com http://www.hetzner.com

Uchikoma 14y ago
Thanks.
imaginator 14y ago
just remember: date -s "`date`" solves it. No reboot, no stopping and starting NTPd.
- hendi_ 14y ago
  date -s "$(LC_ALL=C date)" is the better option which also works on system with a locale != C

metabrew 14y ago

Here's the requisite graph http://i.imgur.com/hsUDE.png http://i.imgur.com/hsUDE.png

dfc 14y ago
where did you get that data?
- atesti 14y ago
  Maybe from here: http://www.heise.de/newsticker/meldung/Schaltsekunden-Bug-in-Linux-verschwendet-Strom-1631325.html http://www.heise.de/newsticker/meldung/Schaltsekunden-Bug-in... Heise.de got it directly from Hetzner, they acknowledge and thank them for providing it in the update of the article
iand 14y ago
Look how spiky the power usage is normally. Is that all the cron jobs set to run at the top of the hour?
- danielweber 14y ago
  Maybe people should get discounts for running their cron jobs at the 50-minute mark.
cellularmitosis 14y ago
/standard rant about the Y-axis not being anchored at zero, which will cause most people to be mislead. UPDATE: here's an illustration. compare this: http://i.imgur.com/f3m25.png http://i.imgur.com/f3m25.png to this: http://i.imgur.com/6z2cS.png http://i.imgur.com/6z2cS.png (those two graphs represent the same data)
- Dylan16807 14y ago
  Really? Who is both going to understand leap seconds, and also look at that graph and think power consumption went up tenfold. I'd much rather see the detail that zoom gives.

etaty 14y ago

OVH https://twitter.com/olesovhcom/status/220142781895671810 https://twitter.com/olesovhcom/status/220142781895671810

deleted 14y ago
[deleted]

ck2 14y ago

If the leap-second does all this, just imagine the 32-bit rollover issue in 2038

forgotusername 14y ago
I'm imagining a whole bunch of bricked "classic" cars
- daeken 14y ago
  I really can't imagine why that would ever happen. What car would ever use a clock for anything essential, instead of a timer? Maybe I'm overestimating the intelligence of car developers, but it seems like the obvious, simple solution.
  - forgotusername 14y ago
    Engine won't start, service due 300 years ago. I'd be willing to bet $10 there's at least a few floating around with real time clocks, no idea what they'd be used for though. If nothing else there's always in-car navigation and entertainment which are definitely candidates, although I suppose less likely to result in a brick.
    - mattdeboard 14y ago
      Sorry, what?
      - forgotusername 14y ago
        I'm engaging in what is commonly known as using one's imagination to make educated guesses at the possible electronic devices within an average motor vehicle (you know, that thing with wheels on it) where a real time clock susceptible to wraparound or overflow might be found, thus resulting in potential malfunction, or in plainer words, "the car won't start, the tuner won't play music, the GPS won't get a signal, this thing is about as useful as a brick".

kodisha 14y ago

Huh? I received my mail in English :)

efutch 14y ago

Perhaps nitpicking, but this is because of a bug in the leap second implementation, not in the leap second per se.

Uchikoma 14y ago
I'd guess it happens by the concurrence of a lep second and a leap second implementation bug. The bug without leap seconds would also not lead to problems.

imaginator 14y ago

Nice graph from Hetzner showing the spike: http://imgur.com/a/ykoup http://imgur.com/a/ykoup