4 ms·

I wonder how those three conditions apply to intuition in software development. The only condition that seems to regularly be unmet is "immediate feedback."

mpweiher 8y ago
...which is why REPLs, fast test suites (1-5 seconds), etc. are so important. It's not just a quantitative thing, you get a qualitatively very very different process happening. People have known this, er, intuitively for some time. Kahneman gives us an handle on why it is: it lets us develop an intuition for the software we are working on.
AstralStorm 8y ago
They do apply reasonably well as long as you're solving similar problems. Even if programming language differs... It is similar with mathematicians. A mathematician may smell a rat in a proof like a programmer a bad pattern in code, but you can only really trust their intuition in their specialty field. E.g. web frontend programmer intuition won't do in backend, desktop UI, mobile, HPC, video game coding, embedded programming, real time applications or safety critical software. This does not preclude polyglots and polymaths or generalists.
tunesmith 8y ago
I think intuition goes downhill as your system increases in complexity. It's harder to develop deep experience or regularity in distributed systems than it is in your monolith that you know inside and out. Feedback can get less immediate, too.
- zwieback 8y ago
  So true. I spent the bulk of my early SW development career in distributed systems that were complex enough that they felt like they had a mind of their own. What I learned back then is that my intuition was worthless. Maybe that's just me but the key to success was always to set aside your expectation of what your code does and instead spend more time instrumenting and collecting data on what was actually going on. This is especially true for completely asynchronous systems like networks or systems that integrate multiple independent subsystems. Things just don't happen in the order you expected when timeouts, aborted operations and retries bubble up and down your OS and application stack. After 30 years I consider myself pretty capable in debugging such systems but one thing I never use is "intuition" as defined in the beginning of the article.
- paulsutter 8y ago
  As the system increases in complexity, you need more and more measurement/monitoring in order to gain intuition for how the system works. For example, watching Ganglia graphs of mapreduce clusters make it 10x easier to tune system parameters.
- onemoresoop 8y ago
  Intuition can give you a sense that the code may be bad, but not necessarily pinpoint the problem in the code
- wenc 8y ago
  Agreed. Also, in complex systems, there are emergent properties that are hard to predict, so any intuitions about system behavior from previous observations aren't always valid. Intuition is a form of pattern-matching on implicit/tacit variables. It only works if there is a pattern to match.
meditativeape 8y ago
I think if you work on a stable piece of software with good test coverage and CI/CD process for a long time, you'd meet all 3 requirements and develop good intuition about that system. How valuable that intuition is and whether it is transferrable to other system is another question.
devonkim 8y ago
We do have a form of feedback in most languages - whether it compiles / is syntactically correct. Expert intuition gets tougher for higher level constructs partly due to both the regularity issue (with a sufficiently fast-moving problem domain what may work one year may be suboptimal later) and feedback loop. Basically, consider it a form of Bayesian learning models. The difference is that with humans if the feedback comes too late, we will reject the signal as we have trouble separating actions and consequences. I can conceptually explain why some commonly bad practices like hard coding variables or not using looping constructs for repetitive statements are to those that keep doing it, but I can’t give quick feedback showing precisely how tough it is to make changes around the code or to even understand it to those that write it because they have developed intuition about their codebase and are now mentally somewhere between Stockholm Syndrome and defensiveness.
Symmetry 8y ago
As a software developer you tend to get very rapid feedback on whether your code is working compared to, say, engineers or scientists. Often you can test a change in code and see if it works almost instantly. Some languages force minute long compile cycles but even that is pretty fast compared to the months a scientist might have to work to test a guess. There might be more ceremony at the workplace but when you were learning to code you probably had immediate feedback.

mpweiher 8y ago

...which is why REPLs, fast test suites (1-5 seconds), etc. are so important. It's not just a quantitative thing, you get a qualitatively very very different process happening. People have known this, er, intuitively for some time. Kahneman gives us an handle on why it is: it lets us develop an intuition for the software we are working on.

AstralStorm 8y ago

They do apply reasonably well as long as you're solving similar problems. Even if programming language differs... It is similar with mathematicians. A mathematician may smell a rat in a proof like a programmer a bad pattern in code, but you can only really trust their intuition in their specialty field. E.g. web frontend programmer intuition won't do in backend, desktop UI, mobile, HPC, video game coding, embedded programming, real time applications or safety critical software. This does not preclude polyglots and polymaths or generalists.

tunesmith 8y ago

I think intuition goes downhill as your system increases in complexity. It's harder to develop deep experience or regularity in distributed systems than it is in your monolith that you know inside and out. Feedback can get less immediate, too.

zwieback 8y ago
So true. I spent the bulk of my early SW development career in distributed systems that were complex enough that they felt like they had a mind of their own. What I learned back then is that my intuition was worthless. Maybe that's just me but the key to success was always to set aside your expectation of what your code does and instead spend more time instrumenting and collecting data on what was actually going on. This is especially true for completely asynchronous systems like networks or systems that integrate multiple independent subsystems. Things just don't happen in the order you expected when timeouts, aborted operations and retries bubble up and down your OS and application stack. After 30 years I consider myself pretty capable in debugging such systems but one thing I never use is "intuition" as defined in the beginning of the article.
paulsutter 8y ago
As the system increases in complexity, you need more and more measurement/monitoring in order to gain intuition for how the system works. For example, watching Ganglia graphs of mapreduce clusters make it 10x easier to tune system parameters.
onemoresoop 8y ago
Intuition can give you a sense that the code may be bad, but not necessarily pinpoint the problem in the code
wenc 8y ago
Agreed. Also, in complex systems, there are emergent properties that are hard to predict, so any intuitions about system behavior from previous observations aren't always valid. Intuition is a form of pattern-matching on implicit/tacit variables. It only works if there is a pattern to match.

meditativeape 8y ago

I think if you work on a stable piece of software with good test coverage and CI/CD process for a long time, you'd meet all 3 requirements and develop good intuition about that system. How valuable that intuition is and whether it is transferrable to other system is another question.

devonkim 8y ago

We do have a form of feedback in most languages - whether it compiles / is syntactically correct. Expert intuition gets tougher for higher level constructs partly due to both the regularity issue (with a sufficiently fast-moving problem domain what may work one year may be suboptimal later) and feedback loop. Basically, consider it a form of Bayesian learning models. The difference is that with humans if the feedback comes too late, we will reject the signal as we have trouble separating actions and consequences. I can conceptually explain why some commonly bad practices like hard coding variables or not using looping constructs for repetitive statements are to those that keep doing it, but I can’t give quick feedback showing precisely how tough it is to make changes around the code or to even understand it to those that write it because they have developed intuition about their codebase and are now mentally somewhere between Stockholm Syndrome and defensiveness.

Symmetry 8y ago

As a software developer you tend to get very rapid feedback on whether your code is working compared to, say, engineers or scientists. Often you can test a change in code and see if it works almost instantly. Some languages force minute long compile cycles but even that is pretty fast compared to the months a scientist might have to work to test a guess. There might be more ceremony at the workplace but when you were learning to code you probably had immediate feedback.