4 ms·

Swap, the sub religion. I've wobbled back and forward on swap. In the early days I used to be annoyed by how much disk space it'd take. (an 8 gig disk with 1 g

Swap, the sub religion. I've wobbled back and forward on swap. In the early days I used to be annoyed by how much disk space it'd take. (an 8 gig disk with 1 gig for swap is too much) I've run an 8 core machine with only 2 gigs of ram, and tried to compile something with boost in it. swap allowed me to kill it, and recover the system. I've run VMs with no swap, some swap and loads. However, what I've never done is actually benchmarked the same workload on machines with no, some and loads of swap. However, I generally defer to Rachel, because Rachel has been there and been bitten by that before. On this point: http://rachelbythebay.com/w/2018/04/28/meta/ http://rachelbythebay.com/w/2018/04/28/meta/ which everyone should read and digest, this remark jumped out: >"This is so easy to test for" If I ever say this and never qualify it, it should read: "haha, yeah, I made that same mistake, that's why I test for it now." The only reason why I am "better"[I hope] than my younger self, is because I've made a bucket load of mistakes before. Some of them are technical, but to be honest a load of them are societal. (as in, bleating like Cassandra and not being able to affect change.) If we, as "engineers" are to grow as a class of people, we have to actually learn from other people's mistakes, not just use them as bias confirmation. This is why I like blog posts where they lay out the problem, outfall, cause, workaround and eventual solution.

fit2rule 7y ago
>benchmarked This is what this story is really about. People not instrumenting their systems, no benchmarks, no real clue how things will perform under load. Swap or not swap: it doesn't matter what you do if you haven't benchmarked the system before you set it up - in this case, its your customers who will tell you, eventually, how your software behaves ...
falsedan 7y ago
> Some of them are technical, but to be honest a load of them are societal. (as in, bleating like Cassandra and not being able to affect change.) This is where I am now, where I'm comfortable making technical mistakes and know enough to avoid the most grevious pitfalls, but struggle to expand the sphere of things affected by my changes beyond a few people or single team/product. Some people get into management for this, but I've seen that management is mostly planning, budgeting, hiring/firing, and non-technical communications.
- healsjnr1 7y ago
  This may seem like dumb wordplay, but while management may seem to be about hiring / firing, budgets, etc, _leadership_ is about _influence_. Influence and leadership are force multipliers. If you can lead people well and influence decisions (be it as a people leader, or a technical leader, or both), your experience can be learnt from by more and more people. There is no doubt that leadership involves doing 'management' more often than not. But it's just one aspect, it's not the end in itself.
  - collyw 7y ago
    Depends on the company.
slacka 7y ago
It's not always about performance. Years ago while traveling I installed Linux to a flash drive to carry my entire OS, apps, and data with me. While I did backup often, I was well aware of the danger that excessive write wear posed in killing my stick, so swap was disabled. To this very day and for the same reason, my Pi also has swap disabled. As a result, I've encountered this flaw the Linux kernel design multiple times. And it is a flaw, as you only have to do the same test on Windows to see how an OS should behave when you run out of RAM: a hiccup followed by your offending tab being terminated by the OOM killer. Locking up the system for 10 min. is just not acceptable.
VectorLock 7y ago
>However, what I've never done is actually benchmarked the same workload on machines with no, some and loads of swap. However, I generally defer to Rachel, because Rachel has been there and been bitten by that before. I have. A well utilized machine is going to absolutely tank once it hits swap. Do you want to engineer your application to be able to cope with two radically different performance regimes, or do you simply want to ensure that your working set stays bounded?
- zaphar 7y ago
  I have too. I've also been in the opposite scenario where not having swap caused run away service kill and restart scenarios. Reading between the lines it looks like Rachel has too. Which is why she advises a "not too much" swap approach. As is usual the answer is not one or the other extreme it's much more nuanced than that.
  - VectorLock 7y ago
    >I've also been in the opposite scenario where not having swap caused run away service kill and restart scenarios. Seems like a good time to just kill it and move on. "Cattle not pets."
    - zaphar 7y ago
      The funny thing about cattle is that they stampede. One host goes down the load increases on the others. They go down too. That cascades until they are all in a kill/restart cycle. I've seen lack of swap cause this. I'm betting Rachel has too.
- chc 7y ago
  If you ensure your working set stays bounded, having swap enabled (say, with swappiness 1) is not a big deal. If you're worried about the performance characteristics of swap, you've already conceded that you aren't confident in the bounds of your working set. So it doesn't seem like the relevant question is "Do we want to have well-bounded memory or use swap?" so much as "Can you tolerate some temporary slowness more easily than a server crash?"

fit2rule 7y ago

>benchmarked This is what this story is really about. People not instrumenting their systems, no benchmarks, no real clue how things will perform under load. Swap or not swap: it doesn't matter what you do if you haven't benchmarked the system before you set it up - in this case, its your customers who will tell you, eventually, how your software behaves ...

falsedan 7y ago

> Some of them are technical, but to be honest a load of them are societal. (as in, bleating like Cassandra and not being able to affect change.) This is where I am now, where I'm comfortable making technical mistakes and know enough to avoid the most grevious pitfalls, but struggle to expand the sphere of things affected by my changes beyond a few people or single team/product. Some people get into management for this, but I've seen that management is mostly planning, budgeting, hiring/firing, and non-technical communications.

healsjnr1 7y ago
This may seem like dumb wordplay, but while management may seem to be about hiring / firing, budgets, etc, _leadership_ is about _influence_. Influence and leadership are force multipliers. If you can lead people well and influence decisions (be it as a people leader, or a technical leader, or both), your experience can be learnt from by more and more people. There is no doubt that leadership involves doing 'management' more often than not. But it's just one aspect, it's not the end in itself.
- collyw 7y ago
  Depends on the company.

slacka 7y ago

It's not always about performance. Years ago while traveling I installed Linux to a flash drive to carry my entire OS, apps, and data with me. While I did backup often, I was well aware of the danger that excessive write wear posed in killing my stick, so swap was disabled. To this very day and for the same reason, my Pi also has swap disabled. As a result, I've encountered this flaw the Linux kernel design multiple times. And it is a flaw, as you only have to do the same test on Windows to see how an OS should behave when you run out of RAM: a hiccup followed by your offending tab being terminated by the OOM killer. Locking up the system for 10 min. is just not acceptable.

VectorLock 7y ago

>However, what I've never done is actually benchmarked the same workload on machines with no, some and loads of swap. However, I generally defer to Rachel, because Rachel has been there and been bitten by that before. I have. A well utilized machine is going to absolutely tank once it hits swap. Do you want to engineer your application to be able to cope with two radically different performance regimes, or do you simply want to ensure that your working set stays bounded?

zaphar 7y ago
I have too. I've also been in the opposite scenario where not having swap caused run away service kill and restart scenarios. Reading between the lines it looks like Rachel has too. Which is why she advises a "not too much" swap approach. As is usual the answer is not one or the other extreme it's much more nuanced than that.
- VectorLock 7y ago
  >I've also been in the opposite scenario where not having swap caused run away service kill and restart scenarios. Seems like a good time to just kill it and move on. "Cattle not pets."
  - zaphar 7y ago
    The funny thing about cattle is that they stampede. One host goes down the load increases on the others. They go down too. That cascades until they are all in a kill/restart cycle. I've seen lack of swap cause this. I'm betting Rachel has too.
chc 7y ago
If you ensure your working set stays bounded, having swap enabled (say, with swappiness 1) is not a big deal. If you're worried about the performance characteristics of swap, you've already conceded that you aren't confident in the bounds of your working set. So it doesn't seem like the relevant question is "Do we want to have well-bounded memory or use swap?" so much as "Can you tolerate some temporary slowness more easily than a server crash?"