5 ms·

I altered the program to first allocate 64G then go through and write on each page of the allocation on my 16G machine, dropped my swap partition. After suitab

by jws 14y ago

I altered the program to first allocate 64G then go through and write on each page of the allocation on my 16G machine, dropped my swap partition. After suitable screwing around to keep gcc from optimizing away the writes, it goes a bit like this… jim@gvdev:~$ ./fail Before Allocated 1 GB Allocated 2 GB ... Allocated 64 GB After Writing in 0 GB Writing in 1 GB ... Writing in 15 GB ... rather long delay here... Write failed: Broken pipe upstairs:~ jim$ <----- bad news here, notice the machine. upstairs:~ jim$ slogin gvdev.XXXXX.net ssh: connect to host gvdev.XXXXX.net port 22: Network is unreachable … so now I have to get back in "going out" clothes, go downstairs, and drive over to gvdev and reboot it.

mileswu 14y ago
Kinda OT, but out of interest, why do you use 'slogin' instead of 'ssh'? Just old habits dieing hard from 'rlogin'?
- jws 14y ago
  Yes.
xentronium 14y ago
So, when there is no memory and no swap space available, instead of crashing, it zombifies whole system? This is weird.
- keeperofdakeys 14y ago
  The linux Out Of Memory killer would have activated (if not disabled). This program derives a 'score' for each program, based on a number of statistics, and kills the process with the highest score. Although it usually makes the right choice, sometimes it can kill innocent programs.
  - jws 14y ago
    It was worse than a dead sshd. Looks like something kernel-ish died badly. There was one more cron message in the syslog after this, but then nothing until I rebooted. Console unresponsive, no ping answering. Nov 2 21:29:33 gvdev kernel: [15115318.546376] fail[17150]: segfault at 7fffe9cbc1e0 ip 00000000004006fe sp 00007fffe9cbb1d0 error 4 in fail[400000+1000]
antirez 14y ago
This is a problem with virtual machines vendors such as Linode where the configured swap space is usually tiny by default... It's much better to provide more swap (At least as much as RAM size) and see the performances start to deteriorate as a sign of alarm of overcommit, instead of having random processes killed.
viraptor 14y ago
Always good to have your sshd on auto-respawn in case OOM makes a bad choice ;) Learned this in a similar scenario.

by jws 14y ago

mileswu 14y ago

Kinda OT, but out of interest, why do you use 'slogin' instead of 'ssh'? Just old habits dieing hard from 'rlogin'?

jws 14y ago
Yes.

xentronium 14y ago

So, when there is no memory and no swap space available, instead of crashing, it zombifies whole system? This is weird.

keeperofdakeys 14y ago
The linux Out Of Memory killer would have activated (if not disabled). This program derives a 'score' for each program, based on a number of statistics, and kills the process with the highest score. Although it usually makes the right choice, sometimes it can kill innocent programs.
- jws 14y ago
  It was worse than a dead sshd. Looks like something kernel-ish died badly. There was one more cron message in the syslog after this, but then nothing until I rebooted. Console unresponsive, no ping answering. Nov 2 21:29:33 gvdev kernel: [15115318.546376] fail[17150]: segfault at 7fffe9cbc1e0 ip 00000000004006fe sp 00007fffe9cbb1d0 error 4 in fail[400000+1000]

antirez 14y ago

This is a problem with virtual machines vendors such as Linode where the configured swap space is usually tiny by default... It's much better to provide more swap (At least as much as RAM size) and see the performances start to deteriorate as a sign of alarm of overcommit, instead of having random processes killed.

viraptor 14y ago

Always good to have your sshd on auto-respawn in case OOM makes a bad choice ;) Learned this in a similar scenario.