22 ms·

Context switching costs more than we give it credit for

590 points by maddynator 6y ago

emodendroket 6y ago
Does it though? I've heard about it so many times that I'm starting to think it cannot possibly be as extremely bad as the never-ending torrent of articles would have me believe.
- skeeter2020 6y ago
  >> never-ending torrent of articles ... that continually interrupt your attention?
bcrl 6y ago
This is why work queues and asynchronous state machines are awesome. All the high performance systems I have worked on avoided having a multitude of threads and have tended towards a thread per CPU model. Use your CPU cycles for things that matter.
wbl 6y ago
I though this would be about trashing cpu state.
cylde_frog 6y ago
>You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story). I thought this was a bit absurd. Good article nonetheless though, I can see how getting into a routine about when you will perform specific tasks throughout the day would be more productive.
oars 6y ago
What makes it worse is the number of times I see people get their work interrupted by a WhatsApp, SMS or Facebook Messenger notification on their phone. That damn pop ding Messenger notification sound is like a dog whistle for many people; instantly dropping what they're doing and picking up their phones to follow this sound. I wonder how this generation will be able to remain focused with all of the distractions that keep popping up around them.
- cylde_frog 6y ago
  This is why I am baffled by Apple watch users. My phone is distracting enough that I make a point of having it turned off and in my backpack when trying to work. I can't imagine having something on my wrist that vibrates whenever an app is trying to get my attention.
  - issamehh 6y ago
    I can't speak for Apple users but my watch actually has the opposite effect. The only notifications I let it show are one's that are determined to be important to look at. Everything else remains on the phone and on mute until it is time to deal with them
  - humanfromearth 6y ago
    Most watch users including myself disable those notifications fairly soon after the initial use. Right now I use my watch for alarms (for sleep), pager duty alerts and fitness/sleep/health related stuff.
    - t-writescode 6y ago
      Do Pager Duty alerts actually work now? Last I tried they ... didn't seem the greatest, but I wasn't on call at the time, so maybe that's the real difference.
  - SulfurHexaFluri 6y ago
    I find it to be the reverse. A tap on my wrist is actually less distracting because I don't have to grab and unlock my phone to see what it is. Its just a quick flick of my eyes down to see if its important. The watch also uses different taps for different kinds of messages so sometimes I know without even looking if I should bother. Previously I would have pulled my phone out of my pocket to look every time it vibrated. Its also a super quick swipe and tap to turn on do not disturb mode on the watch so you get no notifications or calls (there is an exemption list too I think)
  - vonwoodson 6y ago
axxl 6y ago
> Trying to code and listening to music falls under cognitive functions. Naming a variable while listening to your favorite songs' lyrics may result in cognitive overload. You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story). This point in the article lost me pretty quickly. My flow state is drastically improved by music, and I have never ever named a variable after the song name or song artist I'm listening to in my life. Edit: Nor have I ever seen a variable named after a song name or artist. Edit2: NotoriousViewController. TheWeekndTimer. RedHotChiliPointers. Now I want to name my variables like this.
- bcrl 6y ago
  di.fm has served me well over the years while coding. Being able to listen to something without lyrics or a major cognitive load works quite well for me.
  - axxl 6y ago
    I used to listen to them as well! I do prefer less lyrics, or more rhythmic lyrics, but I can do normal music as well, personally. That specific example of why it was a problem just amused me.
- chrisjharris 6y ago
  Do you listen mostly to music with lyrics or accoustic music? I find the latter helpful but the former quite unhelpful - like trying to do math when somebody next to you is shouting randon numbers. But I have a faint recollection of having read research on this where it depended if you were a visual/verbal/etc thinker, and you were more vulnerable to disruptions occurring within your given thinking method.
  - Ataraxy 6y ago
    I listen to all sorts of music and it doesn't really change much. Realistically I tune out of the lyrics while I'm focused on something else anyway.
raldi 6y ago
This can jeopardize domestic tranquility, though. I like to focus entirely on cooking, then clean up in bulk after the meal. My wife likes to clean as she goes. We're both sure the other is wrong.
ehnto 6y ago
I have done some things to eliminate small context switches, as I found even those were a trigger to unfocus. I no longer use two monitors, I use i3 to manage windows so I don't have to move to my mouse, same reason I use vimium. I have all my tools inside the IDE and then a terminal sitting below it for all else. When I have to do frontend implementations, switching between tabs and screens, I can work for much less time and I constsntly lose focus before essentially burning out for the day.
- cylde_frog 6y ago
  Thank for the introduction to vimium. This will change my life
- mobattah 6y ago
  I wish there was i3wm for Mac OS. Would make my life so much better.
deleted 6y ago
[deleted]
WhompingWindows 6y ago
The author states switching from cooking to washing dishes is a huge context switch and a waste of time...I disagree. Everyone who cooks efficiently is constantly switching between tasks, since you're often waiting a few minutes before your next cooking step can occur. Maybe in a professional restaurant where your role is defined, then no, don't switch roles. But in your own kitchen? While my toast is toasting, you know I'm going to get a head start on the dishes, or chop those garnishes, or check in on a second dish, etc. If I cooked by focusing solely on each task until it's done, I'd probably use 50-100% more time to do the same job maybe 10% better.
- mulmen 6y ago
  In covid times I have been teaching myself how to cook, basically from nothing. It takes me forever to do anything. Chopping an onion takes what feels like an eternity. Recipes that claim “5 minutes of prep” are more like 20 minutes for me. It can be very discouraging. What you describe is definitely something that comes with experience. Like using vim effectively. I’m improving but it literally took a once in a century global pandemic to overcome the initial learning curve.
  - ufo 6y ago
    To be honest, the times listed on recipe websites are often too optimistic. That said, for chopping it really helps to have a sharp knife. A dull knife takes longer to do the job and it's also more likely to cause an accident.
    - kadoban 6y ago
      Times listed on recipe websites can go beyond optimisic as well and into just plain being physically impossible. One example I remember: https://slate.com/human-interest/2012/05/how-to-cook-onions-why-recipe-writers-lie-and-lie-about-how-long-they-take-to-caramelize.html https://slate.com/human-interest/2012/05/how-to-cook-onions-...
    - retzkek 6y ago
      > for chopping it really helps to have a sharp knife. A dull knife... is also more likely to cause an accident. This is true, but sharp knives also require good handling skills. If someone has been using dull knives their whole life they will almost certainly cut themselves if given a sharp knife, unless they are also re-trained on how to use them. Watch an episode of "Worst cooks in America" and count how many hands Ann has drawn on with a red Sharpie.
sebringj 6y ago
Music is the only thing that frees my creativity and focus as when I'm lost in it I find I've solved the problem in no time at all because I don't realize the passage of time.
atum47 6y ago
I gave up that illusion of multitasking a long time ago. I tend to devide my task in steps and execute them sequentially. When I bake pizza rolls for instance: I need to make the dough, fill it with ham and cheese and then bake them. Of course that are idle time in between those steps (letting the dough proof and baking, for instance) so I insert a small task in between. Like making coffee or mayonnaise. Looks like I'm doing a whole bunch of things at the same time, but I'm not. I'm making just one task of "preparing a meal". I've noticed context switching is costy at my current job. When I'm working on a feature usually I have to wait for my boss to do a code review. During that time I usually start a new feature. When I have to work in two features simultaneously that's when I make a lot of mistakes (usually small mistakes due to lack of attention). I much rather work at one thing at the time.
foxbarrington 6y ago
It might be common sense that batch processing is faster and more efficient than constantly context switching, but it's important to recognize this isn't always the case. It's not software but you can see an illustration of this with stuffing and addressing envelopes: https://blog.gembaacademy.com/2014/07/29/whats-faster-one-piece-flow-or-mass-production-watch-this-envelope-stuffing-simulation-to-find-out/ https://blog.gembaacademy.com/2014/07/29/whats-faster-one-pi...
totetsu 6y ago
can anyone suggest any resources on how to batch a typical office day?week?
nkurz 6y ago
The best analogy is sorting forks, knives, and spoons when we are putting them in drawers. We can throw all silverware together. But then finding a spoon in that pile will take time whenever we need a spoon. The task changes from getting a spoon (execution) to finding a spoon (identification). And that's a cost we want to avoid. I agree with his overall point about the costs of context switching (and was disappointed that he didn't talk about the low level issues of coordination on multithreaded systems) but this seems like an odd candidate for "best analogy". When I moved to a new house a few years ago, we initially lacked an organizer tray as is typically used to keep silverware sorted in the drawer. Our solution, which we thought was temporary until we bought one, was to unceremoniously and clatteringly dump all the silverware into the same drawer, and to pick out what we wanted at mealtime. Three years later, we're still doing it and have concluded that it's much better system. Sorting silverware was always one of my least favorite parts of putting away washed dishes, and now it is trivial: dump the entire basket from the dishwasher into the drawer, often while grinning with joy. And I don't think I've once become angry at the extra time of searching for a spoon or a fork. Consider, if we're being technical, that the primary advantage of presorting happens when you can sort once and take advantage of the ordering many times. But with silver, it's once-in and once-out. Sorting all at once isn't actually faster, it's just preloading the pain. Also note that you can fit a lot more silverware into a drawer unsorted. Having twice as many spoons and forks might reduce utilization, but silverware is cheap and having more makes finding what you want a lot easier. Anyway, you know the standard advice to measure before you optimize? Given my positive experience with the unsorted method, I wonder if he's even tried the approach he's using as a central negative example. Personally, I'd suggest that more people should try it!
- henrikschroder 6y ago
  1) Buy a dishwasher with a third rack. 2) Sort silverware when you put it into the dishwasher. 3) ??? 4) Profit! Actually, just buy a dishwasher with a third rack regardless, because it's so much better.
polishdude20 6y ago
I've recently been assigned a course to teach where I need to mark lots of assignments. These assignments have 12 short answer questions and each student must do them. Slowest part about marking these is having to context switch for each question. So I'd mark Q1 , then Q2 etc for one assignment, then go over to the next one and start back at Q1. I figured this was super inefficient since I needed to have the answer and context of the question in my mind and keep switching. So I wrote a program to parse all of the questions from all of the reports and present them to me in question batches. So now I see all of Q1 one after the other for each student's and mark those. Then once all of the Q1's are done, I mark all the Q2's. After all the marking is done, I run that through another program that then formats that data to each student so it's back onto their individual report plus any comments or deductions. I find that apart from saving the time to have to open tonnes of docx files, now it takes maybe 5 mins to mark everything.
- mewpmewp2 6y ago
  Yeah that makes sense and it should also reduce any bias if someone ruins their image with first question.
  - MaxBarraclough 6y ago
    It also helps protect against unfairness from the slowly drifting mood of the marker.
    - cutemonster 6y ago
      At the same time, answers by a mediocre student that happens to be sorted just after answers from a horribly bad student, might seem quite brilliant. Or seem not so good, if sorted after the best student. So maybe the answer order should be random.
      - MaxBarraclough 6y ago
        That's true. Ideally then, as you say, there should be a Fisher-Yates shuffle of the papers after each pass through.
PragmaticPulp 6y ago
Context switching has high costs, but it's a mistake to assume those costs are unchangeable. With practice, it's possible to train yourself to get back into focus faster after being interrupted or context switching. It doesn't completely cancel out the costs, but having a strategy to get back into focus helps immensely. The worst mistake I see people make is assuming that focus is something that only strikes serendipitously, and is ruined on the first signs of interruption. Once you accept that focus can, in fact, be cultivated and interruptions can be overcome with deliberate effort to re-focus, it becomes much easier to start training yourself to get back into the flow. The other mistake I see people make is letting minor interruptions spiral into an impromptu 30-minute break of checking HN, browsing Twitter, or other distractions until they feel like getting back to work. Don't like your own habits amplify existing distractions.
timmytokyo 6y ago
Did anyone else get totally confused by this article, thinking it was about operating system context switching? Took me a while to realize he was only talking about cognitive context switches.
- snvzz 6y ago
  I did. I was about to rant that it's the <popular_os> that sucks, and cost on seL4 is an order of magnitude lower.
gumby 6y ago
dang, could you change the title to add "human" at the beginning? I clicked on this thinking it would be about timesharing OS context switching.
gorgoiler 6y ago
The kitchen analogy is apt. Woodwork (shop work, joinery) too. I can think of people in my life for whom cooking without also continuously tidying is anathema. It keeps the deck clear but at a huge efficiency cost. Some people just can’t stand untidiness, yet learning to live with mid-process untidiness — and the associated anxiety — is a big step towards being more productive. It is certainly not easy. When coding, I feel a great anxiety if I am mid solution and have a large amount of the current batch left to complete. It is as if I had taken a gulp of air and dived under the water into a cave. Batching involves being able to commit to living without safety checkpoints, and to be able to get a batch complete in the time allotted. If you don’t, you risk having a large pile of four-fifths completed batches.
- frays 6y ago
  Fantastic analogy.
- madaxe_again 6y ago
  I find it’s more like a kitchen where you start sautéing some onions on the hob, and behind you, the inactive dishwasher spontaneously belches water onto the floor, which you have to deal with now. While you’re disconnecting the water supply, your onions are burning, but just as you get back to them, the power goes out, and you have to go investigate. You come back an hour later, and resume cooking, when somebody knocks on the door, wanting to know how many squirrels can fit in a party balloon. You respond, go back to your onions, but the pan has inexplicably broken in two while you had your back turned. You get a new pan, and then you realise you are out of onions, and it’s 2am, and you go to bed, feeling you’ve accomplished nothing. Most context switching isn’t voluntary, and isn’t controllable.
- twh270 6y ago
  > When coding, I feel a great anxiety if I am mid solution and have a large amount of the current batch left to complete. This doesn't work for all types of coding, but I've found that TDD (test-driven development) helps a _ton_ with eliminating large batches, as well as reducing cognitive load. The trick with TDD though is that it's simple to learn, but hard to master. Another good practice is to reduce the "blast radius" of changes so that you're dealing with one thing at a time. For example, don't introduce a new type (class, interface) as a method parameter while implementing the new type while writing tests for it while changing all of the callers of the method to use the new type. One (small) thing at a time, and prefer to keep everything compiling and working except for that one thing you're working on. As for cooking etc., I differentiate between "tidy" and "clean". Mid-process dirty is common, but I strongly prefer to keep my working area relatively tidy throughout.
shmerl 6y ago
> Trying to code and listening to music falls under cognitive functions. Naming a variable while listening to your favorite songs' lyrics may result in cognitive overload. I don't know. Good music helps me concentrate. Something like: * https://www.youtube.com/watch?v=KUn9SYdPF4A https://www.youtube.com/watch?v=KUn9SYdPF4A * https://www.youtube.com/watch?v=9Qnhn1mCL18 https://www.youtube.com/watch?v=9Qnhn1mCL18
twblalock 6y ago
People talk a lot about the costs of context switching, but not about how to reduce those costs. Instead they talk about reducing context switches, which is often unrealistic. The cost of context switching can be reduced, and reducing it is one of the most important job skills and productivity gains you can get. It will also reduce your frustration. Break up your mental model of what you are doing into smaller tasks. Compartmentalize. Encapsulate. If you are in the middle of something so intricate that you would forget what you were doing if you stepped away from it for a few minutes, persist that state somewhere outside your brain -- write it down! Very few of us are in a position where we can lock ourselves away from others for hours of deep work at a time. So, focus on reducing the impact of the interruptions!
- baby 6y ago
  If we’re engineering ourselves there’s a few problems we need to solve. What you’re talking about is saving the registers/state before switching thread. Another problem is figuring out what’s next to schedule. Visualizing your task as a non-ordered list doesn’t work well. What you need is to order your work with a priority queue. Another problem is interrupts: what are the things that take you out of a task? For example, emails, notifications, etc. Another problem is the noisy neighbor problem: you might get disrupted by other people’s performance, or what they’re working on, or your own lack of progress, etc. The solution of course was given by Feynman: disregard!
  - beckingz 6y ago
    More like a database write ahead log: write down the state before using it, and if you get interrupted you can recover.
    - baby 6y ago
      a WAL is more like "write the state before committing to it"
- redisman 6y ago
  I’m also trying to lessen my ownership over too many systems (small company). If someone else is the go to guy/gal for System C I can just hit them with a 301 Redirect rather than context switching to figure out what they’re asking about and what the implementation details were
antirez 6y ago
Context switch and kernel scheduler implemention details also dominate the networking busy loop tests with one client. The kind of naive benchmark you see often in blog posts where every database tested seems to deliver like 10k ops per sec. You are actually just measuring the latency in the networking ping pong the kernel is able to sustain between two processes, so usually the fastest client implementation wins (because the rest is the same for all the systems, being outside the systems).
arcturus17 6y ago
> You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story) This would happen maybe in one out of a hundred cases if at all, making the statement blatantly false. How can I trust that anything the author says will be minimally rigorous about a topic as complex as cognition when they’re making such outlandish claims, even if they’re being facetious?
- tauntz 6y ago
  That caught my eye as well. I've been developing for 14 years with headphones on for 95% of the time. I don't think I've ever had the problem of naming something based on the music I'm listening to. I guess people are rather different and for some it might be a genuine problem, it's just really surprising for me :)
NiceWayToDoIT 6y ago
Yes, true, but how to explain that to you manager?
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
meerita 6y ago
Here's how I reduced cooking costs to an infimum: switched to carnivore diet, setup a sous vide system and eating as much as raw possible.
jokoon 6y ago
That's why I love down-to-earth data oriented programming, and I cannot stand design patterns, OOP, abstraction etc. I would rather iterate over several intermediate datasets, than going into some tedious other way. A computer is a machine. Why go into abstraction if your project is not complex? It seems developers believe that going into higher abstraction is like using analogies, they think it's like using a new form of language so that other developers might understand what they're doing. Of course there is a minimum of code readability, reusability and maintainability, but in the end, a computer is best as looping linear data.
- lultimouomo 6y ago
  I cannot honestly say if you didn't read the article and think that it is about the cost of context switching for software, or if you are saying that OOP forces the programmer to context-switch and so it decreases his productivity.
GiantSully 6y ago
I have the similar feelings as the author when randomly dealing with tasks one by one. These feelings were gone when I grouped the tasks by some criteria firstly before dealing with them.
kwdc 6y ago
For anyone still unconvinced about batching/chunking, here's a set of tasks: - Cleaning the bathroom - Wiping the poop off the toilet bowl - Polishing the silverware - Vacuuming the car - Shopping - Fixing a syntax error in some code - Washing the car - Eating a hamburger If you don't think batching / chunking is useful, try swapping ramdomly between these tasks and leaving them partially done and resuming. The context switch costs should be abundantly clear. Especially if you add in some travel time. Multitasking relies on tricks and effort to actually work. Not even computers get to multitask (until you add more cores). Context switching is still required. Even going from user to kernel space involves some cost. Listening to music, chewing gum and walking is only multitasking because these are using complementary or different "cores". Real multitasking requires independent pathways. Try reading three books at the same time. Or drinking coca-cola and some fine wine. This is my usual challenge for multitaskers. The books and the drinks wont fully be appreciated if you attempt them via simultaneous multitasking. The context switch isn’t trivial and if not properly done always has negative outcomes. Especially the coke and wine combination. Nasty.
ayroblu 6y ago
See the thing is that I think we give a lot of credit in the cost of context switching, no one talks about the reverse cost, sometimes you have 5 tasks to do in a day and you don't context switch and focus only on the one task at the detriment to other tasks. You ruin your sleep because you're so focused on getting something done and it's only 3am, and so ruin your next day of productivity
nomoreusernames 6y ago
i mean if he had been talking about tabs and stuff he is 100% correct, but making dinner and slapping in a couple of washes in between is easy peasy. i think because its so tactile. flipping tabs it your brain doing all the work. changing topics. but muscle memory in the kitchen makes it go smooth.
Blikkentrekker 6y ago
> My answer is to always batch. Depending on the task's size, batching may take the same amount of time as it will to complete the task. And on a few occasions, batching may take even more time than the actual work. But batching is also a skill that needs to be developed. Absolutely no data is given to back up this assertion, and even if data were given in some research, it would be some kind of average value, selected from a sample size of, say, 20-25 year old, disproportionally male Californiamen, who all studied at the same university, and were all willing to donate their time for some undergraduate research and obtain 50 U.S.D. in return, and the data would already reveal immense spread and variability with clearly around 40% at least heavily increasing productivity by switching, but the article would stress that the average would not, and thus advocate that no man so “context switch”. Yes, practicing batching will make one more efficient at it, but practicing context switching will also make one more efficient at it,and the true reason is that many tasks require one to wait for something else and do nothing. The reason why I'm already washing many of the dishes I cooked with as I cooked is of course simple: there is nothing to do except stirring once in a while at a certain stage in cooking so I might as well use that time to do something else. I also took various sips of my coffee writing this, not because I had nothing else to do, but because it tasted nice.
- collyw 6y ago
  > but practicing context switching will also make one more efficient at it, I am not sure that is true at all. The more I do the more it saps my motivation.
- deleted 6y ago
  [deleted]
- hallgrim 6y ago
  In my experience this is absolutely wrong. Context switching is one of those essentials that you cannot cheat or improve upon. It’s like assuming that if I walk to work often enough, eventually it won’t burn any calories because I got so good at it. The issue of filling waiting time is understandable, but recognizing the cost of switching to various tasks can still be useful in determining how to fill your time. Something that doesn’t take you out of what you were already doing will always be better than something completely unrelated and possibly information-loaded.
ivanmaeder 6y ago
I think context switching is difficult to quantify, and difficult to talk about in a general sense. * I'm sure some people are better at it than others * Some work is easier to switch to and from * Some work demands it—e.g., people in supporting roles that will put up with constant interruptions in order to help others Throw into the mix that some people are better than others at deluding themselves about their abilities to context switch (or handle distractions in general), or who simply struggle focusing on a single task for hours at a time. Personally, I don't find meetings and random conversations too bad. But if I have to completely unload a complicated task from memory and replace it with something else, coming back to the original task can often feel like starting from scratch. Other things to think about: * Figure 2 here— http://www.scielo.org.za/scielo.php?script=sci_arttext&pid=S2224-78902015000300009 http://www.scielo.org.za/scielo.php?script=sci_arttext&pid=S... —from a study showing how "time spent on value-added tasks" drops if working on 3 or more parallel projects * For other reasons (e.g., for faster feedback cycles, reducing overhead), batching isn't always a good idea. Here's a fun video showing that contrary to intuition, batching isn't always faster: https://www.youtube.com/watch?v=dmQORuNFE4c https://www.youtube.com/watch?v=dmQORuNFE4c
JimBlackwood 6y ago
Why do blogs that make such bold claims without any sources, with everyone in the comments disagreeing with these random statements, end up so high on the front page? A few days ago there was also the blog about the UK Covid variant that was just nonsense.
hankchinaski 6y ago
i think this applies only to a specific category of tasks. non-tech people seem so reluctant to understand the cost of context switching for somebody who's doing software
- hallgrim 6y ago
  Yes. I think it has to do with the sheer amount of precise information needing to be kept in working memory to be able to complete the tasks. Switching tasks means lots of bandwidth wasted exchanging that information.
nathias 6y ago
time wasted complaining about context switching > time wasted context switching
praestigiare 6y ago
Rarely have I read an article with such a perfect balance between a premise I agree with an examples I do not. But I think this illustrates something important: what constitutes a context switch is not universal. It depends on what parts of a task you need to think about. In the kitchen, from long practice, I do not have to think about prep, technique, or washing up. There are no decisions I need to make for those tasks. They take very little attention. This frees me to spend my focus on the more abstract goals of what I want to cook, and for me it is not a context switch to wash a bowl while an onion is sauteeing. I can certainly cook while keeping the kitchen clean more efficiently than I could cook and then clean it. But when I was first cooking, it was a context switch to go from chopping an onion to chopping a carrot. Both tasks required my attention and focus.
- BLKNSLVR 6y ago
  I agree most enthusiastically. Upon reading the title I was anticipating an article that would provide a great deal of catharsis, only to find "programming whilst listening to music" and "cleaning dishes whilst cooking"... I'm currently wearing a number of hats in a software implementation project (project management, change management, data management, training, communications) and having to switch between is like coordinating to put down a heavy piece of furniture being carried by six people and then assigning those same six people to help pick up another piece of heavy furniture. Or, more succinctly: Flywheel 1 must be stopped before flywheel 2 can be started. I should write my own article.
ChrisMarshallNY 6y ago
"Water is wet. News at 11." I don't think that context switching costs more than I give it credit for, because I already consider it to be outrageously expensive. I am not a fan of IM platforms, like Slack. I don't listen to music while I'm coding, and I schedule my life around coding sessions; not the other way around. I seem to get a lot done. I was a manager for many years, and I had a few basic philosophies (that I sometimes had to shelve, because, corporation): 1) It was my job to act as an "interruption shield" for my team. If one of my managers, or outside developers wanted information or access that required one of my team stop what they were doing, and attend, then it was my job to be "bad cop," and say "no," or "wait." I often wouldn't even let my team member know that they would have an interruption, until they came up for air. 2) It was my job to know what my team was doing; not their job to tell me what they were doing. This meant no standups (except that I didn't have a choice, as it was policy, so we had useless standups; even though I didn't want them). If I needed regularly scheduled meetings, I would schedule them in a fashion that would reduce the likelihood of interrupting developer "flow." It also meant using things like infrastructure tools, integration testing, and frequent checkins to monitor progress. It was my job to interpret them, and ask questions; not my team's job to send me reports (except that I was required to send a weekly report, so I had to gather reports every Thursday. I worked around that, by letting my team members send a quick, one-paragraph email, then annotating that with details from my observations). 3) I made sure that my team was always able to take care of family and life priorities. I cannot stress enough the importance of this. I had a team of experienced developers with families and medical issues. These would always take priority over their work, and I'd be a coldhearted S.O.B. to refuse to acknowledge that. I wasn't real big on letting them have "me" time, but, if they needed to run out to get their kid out of school in a hurry, I would let them do that. Otherwise, I'd get to hear them spending three hours on the phone, trying to get relatives to step in and help. If they were doing something like selling/buying a house, or having contractors in, I'd support them doing what was needed, but I would insist on making sure they got whatever tasks needed doing out of the way. I also encouraged them to confide in me, issues like divorces, and severe medical issues. I did that by being empathetic, and also closed-mouth. Sometimes, I violated company policy by not informing people of issues with employees, as it would have been completely inappropriate. If the issue ended up causing a disaster (it never did, in over 25 years), then I would take the fall. I learned that lesson, real quick. 4) Classic Management 101: Things go well, I highlight the team. Things go bad, I take the heat. This meant that they felt supported in taking risks, and going down rabbitholes, and not having to spend time in a panic, if the avenue they were exploring wasn't giving good results. 5) The smallest unit of schedule time was 1 day. In extreme circumstances, I would schedule for half days. This allowed context switching to happen in normal cycles. In my experience, this ended up with a very productive team, that worked well together, was extremely motivated, cohesive, honorable and honest, and highly productive. I kept engineers on board for decades. But that was just me. YMMV.
jes 6y ago
The advice to always batch is at odds with the primary advice of systems like the Toyota Production System (TPS) and Lean, which is to strive for one-piece flow in most places in the system under study. A continuous integration system doesn’t batch jobs; it starts working on a job the moment it arrives. Would it be better if CI jobs were arranged into batches that were processed, say, twice a day? I would not want the front desk of a hotel to batch the processing of my request for a room with all the others received in the last hour so that the front desk person can maximize their sense of efficiency. In seeking to maximize the efficiency of each business operation at the local level, do we wind up harming performance of the company as a whole? What is the goal?
- sj4nz 6y ago
  I think this confuses manufacturing/operations and problem solving/analysis. In manufacturing and operations, the thinking has already been "baked" into the activity and you make it faster by adding more people, materials and equipment. (More cashiers, more assembly lines, more factories, more CI compute/memory/storage/bandwidth) With problem solving/analysis context switching would exhaust concentration from the thrashing of unrelated patterns/predictions. If you add more people to a problem solving/analysis context you increase the amount of coordination work for everyone to achieve "shared understanding". If you add a _lot_ of people to a problem solving/analysis context (e.g. open source/crowd sourcing) you throw out the constraint of "shared understanding" and win by probability with many different minds partially thinking about the problem. This takes longer of course-- because the project maintainers then have to sift through the crowd's input. The goal has to be understand the form of work, and adjust the levers accordingly.
  - jes 6y ago
    As I read the article, I had the impression that what the author was describing was the kind of activities that they would batch, for example, reviewing pull requests. That sounds more like part of the work involved in manufacturing software than what you refer to as problem solving / analysis. What if the author decided to batch the processing of pull requests so that they processed all of their outstanding PRs once per week? Would that be a good thing for the rest of the team? What if you're trying to deliver an important fix and your PR is stuck in queue waiting for Friday to roll around before the author gets around to review it? The late Colonel John Boyd made important observations as to the competitive advantages that accrue to organizations that can move quickly and well-enough to accomplish the mission. I certainly could be wrong, but batching everything, as the author seems to be suggesting, sounds like a recipe for an organization that is slow as molasses in January, but also populated with people who see themselves as being highly efficient. What am I not seeing?
foolinaround 6y ago
It would be great if there is a follow-through research article that quantifies the cost of context switching for developers in an open office plan. There may be some real-estate cost savings too... these 2 numbers need to be compared...
MrYellowP 6y ago
I'm surprised this isn't common knowledge.
pgroves 6y ago
Batching things into 'contexts' is the premise of Getting Things Done by David Allen. It was popular in the early 2000s. My guess is that the older engineer that told the author to batch things was referencing it [1]. It's a business book based on aggressively making and prioritizing todo lists and then working on them in contexts like 'email', 'phone calls', etc. [1] https://en.wikipedia.org/wiki/Getting_Things_Done https://en.wikipedia.org/wiki/Getting_Things_Done
IndexPointer 6y ago
I'm not sure I understand what he means by batching by function. Is it dividing by work type (eg coding, responding emails, writing documents, etc) and doing all the tasks in one type before going to the next?

590 points by maddynator 6y ago

Does it though? I've heard about it so many times that I'm starting to think it cannot possibly be as extremely bad as the never-ending torrent of articles would have me believe.

skeeter2020 6y ago
>> never-ending torrent of articles ... that continually interrupt your attention?

bcrl 6y ago

This is why work queues and asynchronous state machines are awesome. All the high performance systems I have worked on avoided having a multitude of threads and have tended towards a thread per CPU model. Use your CPU cycles for things that matter.

wbl 6y ago

I though this would be about trashing cpu state.

cylde_frog 6y ago

>You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story). I thought this was a bit absurd. Good article nonetheless though, I can see how getting into a routine about when you will perform specific tasks throughout the day would be more productive.

oars 6y ago

What makes it worse is the number of times I see people get their work interrupted by a WhatsApp, SMS or Facebook Messenger notification on their phone. That damn pop ding Messenger notification sound is like a dog whistle for many people; instantly dropping what they're doing and picking up their phones to follow this sound. I wonder how this generation will be able to remain focused with all of the distractions that keep popping up around them.

cylde_frog 6y ago
This is why I am baffled by Apple watch users. My phone is distracting enough that I make a point of having it turned off and in my backpack when trying to work. I can't imagine having something on my wrist that vibrates whenever an app is trying to get my attention.
- issamehh 6y ago
  I can't speak for Apple users but my watch actually has the opposite effect. The only notifications I let it show are one's that are determined to be important to look at. Everything else remains on the phone and on mute until it is time to deal with them
- humanfromearth 6y ago
  Most watch users including myself disable those notifications fairly soon after the initial use. Right now I use my watch for alarms (for sleep), pager duty alerts and fitness/sleep/health related stuff.
  - t-writescode 6y ago
    Do Pager Duty alerts actually work now? Last I tried they ... didn't seem the greatest, but I wasn't on call at the time, so maybe that's the real difference.
- SulfurHexaFluri 6y ago
  I find it to be the reverse. A tap on my wrist is actually less distracting because I don't have to grab and unlock my phone to see what it is. Its just a quick flick of my eyes down to see if its important. The watch also uses different taps for different kinds of messages so sometimes I know without even looking if I should bother. Previously I would have pulled my phone out of my pocket to look every time it vibrated. Its also a super quick swipe and tap to turn on do not disturb mode on the watch so you get no notifications or calls (there is an exemption list too I think)
- vonwoodson 6y ago

axxl 6y ago

> Trying to code and listening to music falls under cognitive functions. Naming a variable while listening to your favorite songs' lyrics may result in cognitive overload. You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story). This point in the article lost me pretty quickly. My flow state is drastically improved by music, and I have never ever named a variable after the song name or song artist I'm listening to in my life. Edit: Nor have I ever seen a variable named after a song name or artist. Edit2: NotoriousViewController. TheWeekndTimer. RedHotChiliPointers. Now I want to name my variables like this.

bcrl 6y ago
di.fm has served me well over the years while coding. Being able to listen to something without lyrics or a major cognitive load works quite well for me.
- axxl 6y ago
  I used to listen to them as well! I do prefer less lyrics, or more rhythmic lyrics, but I can do normal music as well, personally. That specific example of why it was a problem just amused me.
chrisjharris 6y ago
Do you listen mostly to music with lyrics or accoustic music? I find the latter helpful but the former quite unhelpful - like trying to do math when somebody next to you is shouting randon numbers. But I have a faint recollection of having read research on this where it depended if you were a visual/verbal/etc thinker, and you were more vulnerable to disruptions occurring within your given thinking method.
- Ataraxy 6y ago
  I listen to all sorts of music and it doesn't really change much. Realistically I tune out of the lyrics while I'm focused on something else anyway.

raldi 6y ago

This can jeopardize domestic tranquility, though. I like to focus entirely on cooking, then clean up in bulk after the meal. My wife likes to clean as she goes. We're both sure the other is wrong.

ehnto 6y ago

I have done some things to eliminate small context switches, as I found even those were a trigger to unfocus. I no longer use two monitors, I use i3 to manage windows so I don't have to move to my mouse, same reason I use vimium. I have all my tools inside the IDE and then a terminal sitting below it for all else. When I have to do frontend implementations, switching between tabs and screens, I can work for much less time and I constsntly lose focus before essentially burning out for the day.

cylde_frog 6y ago
Thank for the introduction to vimium. This will change my life
mobattah 6y ago
I wish there was i3wm for Mac OS. Would make my life so much better.

deleted 6y ago

[deleted]

WhompingWindows 6y ago

The author states switching from cooking to washing dishes is a huge context switch and a waste of time...I disagree. Everyone who cooks efficiently is constantly switching between tasks, since you're often waiting a few minutes before your next cooking step can occur. Maybe in a professional restaurant where your role is defined, then no, don't switch roles. But in your own kitchen? While my toast is toasting, you know I'm going to get a head start on the dishes, or chop those garnishes, or check in on a second dish, etc. If I cooked by focusing solely on each task until it's done, I'd probably use 50-100% more time to do the same job maybe 10% better.

mulmen 6y ago
In covid times I have been teaching myself how to cook, basically from nothing. It takes me forever to do anything. Chopping an onion takes what feels like an eternity. Recipes that claim “5 minutes of prep” are more like 20 minutes for me. It can be very discouraging. What you describe is definitely something that comes with experience. Like using vim effectively. I’m improving but it literally took a once in a century global pandemic to overcome the initial learning curve.
- ufo 6y ago
  To be honest, the times listed on recipe websites are often too optimistic. That said, for chopping it really helps to have a sharp knife. A dull knife takes longer to do the job and it's also more likely to cause an accident.
  - kadoban 6y ago
    Times listed on recipe websites can go beyond optimisic as well and into just plain being physically impossible. One example I remember: https://slate.com/human-interest/2012/05/how-to-cook-onions-why-recipe-writers-lie-and-lie-about-how-long-they-take-to-caramelize.html https://slate.com/human-interest/2012/05/how-to-cook-onions-...
  - retzkek 6y ago
    > for chopping it really helps to have a sharp knife. A dull knife... is also more likely to cause an accident. This is true, but sharp knives also require good handling skills. If someone has been using dull knives their whole life they will almost certainly cut themselves if given a sharp knife, unless they are also re-trained on how to use them. Watch an episode of "Worst cooks in America" and count how many hands Ann has drawn on with a red Sharpie.

sebringj 6y ago

Music is the only thing that frees my creativity and focus as when I'm lost in it I find I've solved the problem in no time at all because I don't realize the passage of time.

atum47 6y ago

I gave up that illusion of multitasking a long time ago. I tend to devide my task in steps and execute them sequentially. When I bake pizza rolls for instance: I need to make the dough, fill it with ham and cheese and then bake them. Of course that are idle time in between those steps (letting the dough proof and baking, for instance) so I insert a small task in between. Like making coffee or mayonnaise. Looks like I'm doing a whole bunch of things at the same time, but I'm not. I'm making just one task of "preparing a meal". I've noticed context switching is costy at my current job. When I'm working on a feature usually I have to wait for my boss to do a code review. During that time I usually start a new feature. When I have to work in two features simultaneously that's when I make a lot of mistakes (usually small mistakes due to lack of attention). I much rather work at one thing at the time.

foxbarrington 6y ago

It might be common sense that batch processing is faster and more efficient than constantly context switching, but it's important to recognize this isn't always the case. It's not software but you can see an illustration of this with stuffing and addressing envelopes: https://blog.gembaacademy.com/2014/07/29/whats-faster-one-piece-flow-or-mass-production-watch-this-envelope-stuffing-simulation-to-find-out/ https://blog.gembaacademy.com/2014/07/29/whats-faster-one-pi...

totetsu 6y ago

can anyone suggest any resources on how to batch a typical office day?week?

nkurz 6y ago

The best analogy is sorting forks, knives, and spoons when we are putting them in drawers. We can throw all silverware together. But then finding a spoon in that pile will take time whenever we need a spoon. The task changes from getting a spoon (execution) to finding a spoon (identification). And that's a cost we want to avoid. I agree with his overall point about the costs of context switching (and was disappointed that he didn't talk about the low level issues of coordination on multithreaded systems) but this seems like an odd candidate for "best analogy". When I moved to a new house a few years ago, we initially lacked an organizer tray as is typically used to keep silverware sorted in the drawer. Our solution, which we thought was temporary until we bought one, was to unceremoniously and clatteringly dump all the silverware into the same drawer, and to pick out what we wanted at mealtime. Three years later, we're still doing it and have concluded that it's much better system. Sorting silverware was always one of my least favorite parts of putting away washed dishes, and now it is trivial: dump the entire basket from the dishwasher into the drawer, often while grinning with joy. And I don't think I've once become angry at the extra time of searching for a spoon or a fork. Consider, if we're being technical, that the primary advantage of presorting happens when you can sort once and take advantage of the ordering many times. But with silver, it's once-in and once-out. Sorting all at once isn't actually faster, it's just preloading the pain. Also note that you can fit a lot more silverware into a drawer unsorted. Having twice as many spoons and forks might reduce utilization, but silverware is cheap and having more makes finding what you want a lot easier. Anyway, you know the standard advice to measure before you optimize? Given my positive experience with the unsorted method, I wonder if he's even tried the approach he's using as a central negative example. Personally, I'd suggest that more people should try it!

henrikschroder 6y ago
1) Buy a dishwasher with a third rack. 2) Sort silverware when you put it into the dishwasher. 3) ??? 4) Profit! Actually, just buy a dishwasher with a third rack regardless, because it's so much better.

polishdude20 6y ago

I've recently been assigned a course to teach where I need to mark lots of assignments. These assignments have 12 short answer questions and each student must do them. Slowest part about marking these is having to context switch for each question. So I'd mark Q1 , then Q2 etc for one assignment, then go over to the next one and start back at Q1. I figured this was super inefficient since I needed to have the answer and context of the question in my mind and keep switching. So I wrote a program to parse all of the questions from all of the reports and present them to me in question batches. So now I see all of Q1 one after the other for each student's and mark those. Then once all of the Q1's are done, I mark all the Q2's. After all the marking is done, I run that through another program that then formats that data to each student so it's back onto their individual report plus any comments or deductions. I find that apart from saving the time to have to open tonnes of docx files, now it takes maybe 5 mins to mark everything.

mewpmewp2 6y ago
Yeah that makes sense and it should also reduce any bias if someone ruins their image with first question.
- MaxBarraclough 6y ago
  It also helps protect against unfairness from the slowly drifting mood of the marker.
  - cutemonster 6y ago
    At the same time, answers by a mediocre student that happens to be sorted just after answers from a horribly bad student, might seem quite brilliant. Or seem not so good, if sorted after the best student. So maybe the answer order should be random.
    - MaxBarraclough 6y ago
      That's true. Ideally then, as you say, there should be a Fisher-Yates shuffle of the papers after each pass through.

PragmaticPulp 6y ago

Context switching has high costs, but it's a mistake to assume those costs are unchangeable. With practice, it's possible to train yourself to get back into focus faster after being interrupted or context switching. It doesn't completely cancel out the costs, but having a strategy to get back into focus helps immensely. The worst mistake I see people make is assuming that focus is something that only strikes serendipitously, and is ruined on the first signs of interruption. Once you accept that focus can, in fact, be cultivated and interruptions can be overcome with deliberate effort to re-focus, it becomes much easier to start training yourself to get back into the flow. The other mistake I see people make is letting minor interruptions spiral into an impromptu 30-minute break of checking HN, browsing Twitter, or other distractions until they feel like getting back to work. Don't like your own habits amplify existing distractions.

timmytokyo 6y ago

Did anyone else get totally confused by this article, thinking it was about operating system context switching? Took me a while to realize he was only talking about cognitive context switches.

snvzz 6y ago
I did. I was about to rant that it's the <popular_os> that sucks, and cost on seL4 is an order of magnitude lower.

gumby 6y ago

dang, could you change the title to add "human" at the beginning? I clicked on this thinking it would be about timesharing OS context switching.

gorgoiler 6y ago

The kitchen analogy is apt. Woodwork (shop work, joinery) too. I can think of people in my life for whom cooking without also continuously tidying is anathema. It keeps the deck clear but at a huge efficiency cost. Some people just can’t stand untidiness, yet learning to live with mid-process untidiness — and the associated anxiety — is a big step towards being more productive. It is certainly not easy. When coding, I feel a great anxiety if I am mid solution and have a large amount of the current batch left to complete. It is as if I had taken a gulp of air and dived under the water into a cave. Batching involves being able to commit to living without safety checkpoints, and to be able to get a batch complete in the time allotted. If you don’t, you risk having a large pile of four-fifths completed batches.

frays 6y ago
Fantastic analogy.
madaxe_again 6y ago
I find it’s more like a kitchen where you start sautéing some onions on the hob, and behind you, the inactive dishwasher spontaneously belches water onto the floor, which you have to deal with now. While you’re disconnecting the water supply, your onions are burning, but just as you get back to them, the power goes out, and you have to go investigate. You come back an hour later, and resume cooking, when somebody knocks on the door, wanting to know how many squirrels can fit in a party balloon. You respond, go back to your onions, but the pan has inexplicably broken in two while you had your back turned. You get a new pan, and then you realise you are out of onions, and it’s 2am, and you go to bed, feeling you’ve accomplished nothing. Most context switching isn’t voluntary, and isn’t controllable.
twh270 6y ago
> When coding, I feel a great anxiety if I am mid solution and have a large amount of the current batch left to complete. This doesn't work for all types of coding, but I've found that TDD (test-driven development) helps a _ton_ with eliminating large batches, as well as reducing cognitive load. The trick with TDD though is that it's simple to learn, but hard to master. Another good practice is to reduce the "blast radius" of changes so that you're dealing with one thing at a time. For example, don't introduce a new type (class, interface) as a method parameter while implementing the new type while writing tests for it while changing all of the callers of the method to use the new type. One (small) thing at a time, and prefer to keep everything compiling and working except for that one thing you're working on. As for cooking etc., I differentiate between "tidy" and "clean". Mid-process dirty is common, but I strongly prefer to keep my working area relatively tidy throughout.

shmerl 6y ago

> Trying to code and listening to music falls under cognitive functions. Naming a variable while listening to your favorite songs' lyrics may result in cognitive overload. I don't know. Good music helps me concentrate. Something like: * https://www.youtube.com/watch?v=KUn9SYdPF4A https://www.youtube.com/watch?v=KUn9SYdPF4A * https://www.youtube.com/watch?v=9Qnhn1mCL18 https://www.youtube.com/watch?v=9Qnhn1mCL18

twblalock 6y ago

People talk a lot about the costs of context switching, but not about how to reduce those costs. Instead they talk about reducing context switches, which is often unrealistic. The cost of context switching can be reduced, and reducing it is one of the most important job skills and productivity gains you can get. It will also reduce your frustration. Break up your mental model of what you are doing into smaller tasks. Compartmentalize. Encapsulate. If you are in the middle of something so intricate that you would forget what you were doing if you stepped away from it for a few minutes, persist that state somewhere outside your brain -- write it down! Very few of us are in a position where we can lock ourselves away from others for hours of deep work at a time. So, focus on reducing the impact of the interruptions!

baby 6y ago
If we’re engineering ourselves there’s a few problems we need to solve. What you’re talking about is saving the registers/state before switching thread. Another problem is figuring out what’s next to schedule. Visualizing your task as a non-ordered list doesn’t work well. What you need is to order your work with a priority queue. Another problem is interrupts: what are the things that take you out of a task? For example, emails, notifications, etc. Another problem is the noisy neighbor problem: you might get disrupted by other people’s performance, or what they’re working on, or your own lack of progress, etc. The solution of course was given by Feynman: disregard!
- beckingz 6y ago
  More like a database write ahead log: write down the state before using it, and if you get interrupted you can recover.
  - baby 6y ago
    a WAL is more like "write the state before committing to it"
redisman 6y ago
I’m also trying to lessen my ownership over too many systems (small company). If someone else is the go to guy/gal for System C I can just hit them with a 301 Redirect rather than context switching to figure out what they’re asking about and what the implementation details were

antirez 6y ago

Context switch and kernel scheduler implemention details also dominate the networking busy loop tests with one client. The kind of naive benchmark you see often in blog posts where every database tested seems to deliver like 10k ops per sec. You are actually just measuring the latency in the networking ping pong the kernel is able to sustain between two processes, so usually the fastest client implementation wins (because the rest is the same for all the systems, being outside the systems).

arcturus17 6y ago

> You are more likely to name a variable after the singers/songs name than what the variable is supposed to reflect (true story) This would happen maybe in one out of a hundred cases if at all, making the statement blatantly false. How can I trust that anything the author says will be minimally rigorous about a topic as complex as cognition when they’re making such outlandish claims, even if they’re being facetious?

tauntz 6y ago
That caught my eye as well. I've been developing for 14 years with headphones on for 95% of the time. I don't think I've ever had the problem of naming something based on the music I'm listening to. I guess people are rather different and for some it might be a genuine problem, it's just really surprising for me :)

NiceWayToDoIT 6y ago

Yes, true, but how to explain that to you manager?

deleted 6y ago
[deleted]
deleted 6y ago
[deleted]

meerita 6y ago

Here's how I reduced cooking costs to an infimum: switched to carnivore diet, setup a sous vide system and eating as much as raw possible.

jokoon 6y ago

That's why I love down-to-earth data oriented programming, and I cannot stand design patterns, OOP, abstraction etc. I would rather iterate over several intermediate datasets, than going into some tedious other way. A computer is a machine. Why go into abstraction if your project is not complex? It seems developers believe that going into higher abstraction is like using analogies, they think it's like using a new form of language so that other developers might understand what they're doing. Of course there is a minimum of code readability, reusability and maintainability, but in the end, a computer is best as looping linear data.

lultimouomo 6y ago
I cannot honestly say if you didn't read the article and think that it is about the cost of context switching for software, or if you are saying that OOP forces the programmer to context-switch and so it decreases his productivity.

GiantSully 6y ago

I have the similar feelings as the author when randomly dealing with tasks one by one. These feelings were gone when I grouped the tasks by some criteria firstly before dealing with them.

kwdc 6y ago

For anyone still unconvinced about batching/chunking, here's a set of tasks: - Cleaning the bathroom - Wiping the poop off the toilet bowl - Polishing the silverware - Vacuuming the car - Shopping - Fixing a syntax error in some code - Washing the car - Eating a hamburger If you don't think batching / chunking is useful, try swapping ramdomly between these tasks and leaving them partially done and resuming. The context switch costs should be abundantly clear. Especially if you add in some travel time. Multitasking relies on tricks and effort to actually work. Not even computers get to multitask (until you add more cores). Context switching is still required. Even going from user to kernel space involves some cost. Listening to music, chewing gum and walking is only multitasking because these are using complementary or different "cores". Real multitasking requires independent pathways. Try reading three books at the same time. Or drinking coca-cola and some fine wine. This is my usual challenge for multitaskers. The books and the drinks wont fully be appreciated if you attempt them via simultaneous multitasking. The context switch isn’t trivial and if not properly done always has negative outcomes. Especially the coke and wine combination. Nasty.

ayroblu 6y ago

See the thing is that I think we give a lot of credit in the cost of context switching, no one talks about the reverse cost, sometimes you have 5 tasks to do in a day and you don't context switch and focus only on the one task at the detriment to other tasks. You ruin your sleep because you're so focused on getting something done and it's only 3am, and so ruin your next day of productivity

nomoreusernames 6y ago

i mean if he had been talking about tabs and stuff he is 100% correct, but making dinner and slapping in a couple of washes in between is easy peasy. i think because its so tactile. flipping tabs it your brain doing all the work. changing topics. but muscle memory in the kitchen makes it go smooth.

Blikkentrekker 6y ago

> My answer is to always batch. Depending on the task's size, batching may take the same amount of time as it will to complete the task. And on a few occasions, batching may take even more time than the actual work. But batching is also a skill that needs to be developed. Absolutely no data is given to back up this assertion, and even if data were given in some research, it would be some kind of average value, selected from a sample size of, say, 20-25 year old, disproportionally male Californiamen, who all studied at the same university, and were all willing to donate their time for some undergraduate research and obtain 50 U.S.D. in return, and the data would already reveal immense spread and variability with clearly around 40% at least heavily increasing productivity by switching, but the article would stress that the average would not, and thus advocate that no man so “context switch”. Yes, practicing batching will make one more efficient at it, but practicing context switching will also make one more efficient at it,and the true reason is that many tasks require one to wait for something else and do nothing. The reason why I'm already washing many of the dishes I cooked with as I cooked is of course simple: there is nothing to do except stirring once in a while at a certain stage in cooking so I might as well use that time to do something else. I also took various sips of my coffee writing this, not because I had nothing else to do, but because it tasted nice.

collyw 6y ago
> but practicing context switching will also make one more efficient at it, I am not sure that is true at all. The more I do the more it saps my motivation.
deleted 6y ago
[deleted]
hallgrim 6y ago
In my experience this is absolutely wrong. Context switching is one of those essentials that you cannot cheat or improve upon. It’s like assuming that if I walk to work often enough, eventually it won’t burn any calories because I got so good at it. The issue of filling waiting time is understandable, but recognizing the cost of switching to various tasks can still be useful in determining how to fill your time. Something that doesn’t take you out of what you were already doing will always be better than something completely unrelated and possibly information-loaded.

ivanmaeder 6y ago

I think context switching is difficult to quantify, and difficult to talk about in a general sense. * I'm sure some people are better at it than others * Some work is easier to switch to and from * Some work demands it—e.g., people in supporting roles that will put up with constant interruptions in order to help others Throw into the mix that some people are better than others at deluding themselves about their abilities to context switch (or handle distractions in general), or who simply struggle focusing on a single task for hours at a time. Personally, I don't find meetings and random conversations too bad. But if I have to completely unload a complicated task from memory and replace it with something else, coming back to the original task can often feel like starting from scratch. Other things to think about: * Figure 2 here— http://www.scielo.org.za/scielo.php?script=sci_arttext&pid=S2224-78902015000300009 http://www.scielo.org.za/scielo.php?script=sci_arttext&pid=S... —from a study showing how "time spent on value-added tasks" drops if working on 3 or more parallel projects * For other reasons (e.g., for faster feedback cycles, reducing overhead), batching isn't always a good idea. Here's a fun video showing that contrary to intuition, batching isn't always faster: https://www.youtube.com/watch?v=dmQORuNFE4c https://www.youtube.com/watch?v=dmQORuNFE4c

JimBlackwood 6y ago

Why do blogs that make such bold claims without any sources, with everyone in the comments disagreeing with these random statements, end up so high on the front page? A few days ago there was also the blog about the UK Covid variant that was just nonsense.

hankchinaski 6y ago

i think this applies only to a specific category of tasks. non-tech people seem so reluctant to understand the cost of context switching for somebody who's doing software

hallgrim 6y ago
Yes. I think it has to do with the sheer amount of precise information needing to be kept in working memory to be able to complete the tasks. Switching tasks means lots of bandwidth wasted exchanging that information.

nathias 6y ago

time wasted complaining about context switching > time wasted context switching

praestigiare 6y ago

Rarely have I read an article with such a perfect balance between a premise I agree with an examples I do not. But I think this illustrates something important: what constitutes a context switch is not universal. It depends on what parts of a task you need to think about. In the kitchen, from long practice, I do not have to think about prep, technique, or washing up. There are no decisions I need to make for those tasks. They take very little attention. This frees me to spend my focus on the more abstract goals of what I want to cook, and for me it is not a context switch to wash a bowl while an onion is sauteeing. I can certainly cook while keeping the kitchen clean more efficiently than I could cook and then clean it. But when I was first cooking, it was a context switch to go from chopping an onion to chopping a carrot. Both tasks required my attention and focus.

BLKNSLVR 6y ago
I agree most enthusiastically. Upon reading the title I was anticipating an article that would provide a great deal of catharsis, only to find "programming whilst listening to music" and "cleaning dishes whilst cooking"... I'm currently wearing a number of hats in a software implementation project (project management, change management, data management, training, communications) and having to switch between is like coordinating to put down a heavy piece of furniture being carried by six people and then assigning those same six people to help pick up another piece of heavy furniture. Or, more succinctly: Flywheel 1 must be stopped before flywheel 2 can be started. I should write my own article.

ChrisMarshallNY 6y ago

"Water is wet. News at 11." I don't think that context switching costs more than I give it credit for, because I already consider it to be outrageously expensive. I am not a fan of IM platforms, like Slack. I don't listen to music while I'm coding, and I schedule my life around coding sessions; not the other way around. I seem to get a lot done. I was a manager for many years, and I had a few basic philosophies (that I sometimes had to shelve, because, corporation): 1) It was my job to act as an "interruption shield" for my team. If one of my managers, or outside developers wanted information or access that required one of my team stop what they were doing, and attend, then it was my job to be "bad cop," and say "no," or "wait." I often wouldn't even let my team member know that they would have an interruption, until they came up for air. 2) It was my job to know what my team was doing; not their job to tell me what they were doing. This meant no standups (except that I didn't have a choice, as it was policy, so we had useless standups; even though I didn't want them). If I needed regularly scheduled meetings, I would schedule them in a fashion that would reduce the likelihood of interrupting developer "flow." It also meant using things like infrastructure tools, integration testing, and frequent checkins to monitor progress. It was my job to interpret them, and ask questions; not my team's job to send me reports (except that I was required to send a weekly report, so I had to gather reports every Thursday. I worked around that, by letting my team members send a quick, one-paragraph email, then annotating that with details from my observations). 3) I made sure that my team was always able to take care of family and life priorities. I cannot stress enough the importance of this. I had a team of experienced developers with families and medical issues. These would always take priority over their work, and I'd be a coldhearted S.O.B. to refuse to acknowledge that. I wasn't real big on letting them have "me" time, but, if they needed to run out to get their kid out of school in a hurry, I would let them do that. Otherwise, I'd get to hear them spending three hours on the phone, trying to get relatives to step in and help. If they were doing something like selling/buying a house, or having contractors in, I'd support them doing what was needed, but I would insist on making sure they got whatever tasks needed doing out of the way. I also encouraged them to confide in me, issues like divorces, and severe medical issues. I did that by being empathetic, and also closed-mouth. Sometimes, I violated company policy by not informing people of issues with employees, as it would have been completely inappropriate. If the issue ended up causing a disaster (it never did, in over 25 years), then I would take the fall. I learned that lesson, real quick. 4) Classic Management 101: Things go well, I highlight the team. Things go bad, I take the heat. This meant that they felt supported in taking risks, and going down rabbitholes, and not having to spend time in a panic, if the avenue they were exploring wasn't giving good results. 5) The smallest unit of schedule time was 1 day. In extreme circumstances, I would schedule for half days. This allowed context switching to happen in normal cycles. In my experience, this ended up with a very productive team, that worked well together, was extremely motivated, cohesive, honorable and honest, and highly productive. I kept engineers on board for decades. But that was just me. YMMV.

jes 6y ago

The advice to always batch is at odds with the primary advice of systems like the Toyota Production System (TPS) and Lean, which is to strive for one-piece flow in most places in the system under study. A continuous integration system doesn’t batch jobs; it starts working on a job the moment it arrives. Would it be better if CI jobs were arranged into batches that were processed, say, twice a day? I would not want the front desk of a hotel to batch the processing of my request for a room with all the others received in the last hour so that the front desk person can maximize their sense of efficiency. In seeking to maximize the efficiency of each business operation at the local level, do we wind up harming performance of the company as a whole? What is the goal?

sj4nz 6y ago
I think this confuses manufacturing/operations and problem solving/analysis. In manufacturing and operations, the thinking has already been "baked" into the activity and you make it faster by adding more people, materials and equipment. (More cashiers, more assembly lines, more factories, more CI compute/memory/storage/bandwidth) With problem solving/analysis context switching would exhaust concentration from the thrashing of unrelated patterns/predictions. If you add more people to a problem solving/analysis context you increase the amount of coordination work for everyone to achieve "shared understanding". If you add a _lot_ of people to a problem solving/analysis context (e.g. open source/crowd sourcing) you throw out the constraint of "shared understanding" and win by probability with many different minds partially thinking about the problem. This takes longer of course-- because the project maintainers then have to sift through the crowd's input. The goal has to be understand the form of work, and adjust the levers accordingly.
- jes 6y ago
  As I read the article, I had the impression that what the author was describing was the kind of activities that they would batch, for example, reviewing pull requests. That sounds more like part of the work involved in manufacturing software than what you refer to as problem solving / analysis. What if the author decided to batch the processing of pull requests so that they processed all of their outstanding PRs once per week? Would that be a good thing for the rest of the team? What if you're trying to deliver an important fix and your PR is stuck in queue waiting for Friday to roll around before the author gets around to review it? The late Colonel John Boyd made important observations as to the competitive advantages that accrue to organizations that can move quickly and well-enough to accomplish the mission. I certainly could be wrong, but batching everything, as the author seems to be suggesting, sounds like a recipe for an organization that is slow as molasses in January, but also populated with people who see themselves as being highly efficient. What am I not seeing?

foolinaround 6y ago

It would be great if there is a follow-through research article that quantifies the cost of context switching for developers in an open office plan. There may be some real-estate cost savings too... these 2 numbers need to be compared...

MrYellowP 6y ago

I'm surprised this isn't common knowledge.

pgroves 6y ago

Batching things into 'contexts' is the premise of Getting Things Done by David Allen. It was popular in the early 2000s. My guess is that the older engineer that told the author to batch things was referencing it [1]. It's a business book based on aggressively making and prioritizing todo lists and then working on them in contexts like 'email', 'phone calls', etc. [1] https://en.wikipedia.org/wiki/Getting_Things_Done https://en.wikipedia.org/wiki/Getting_Things_Done

IndexPointer 6y ago

I'm not sure I understand what he means by batching by function. Is it dividing by work type (eg coding, responding emails, writing documents, etc) and doing all the tasks in one type before going to the next?