5 ms·

Tacit knowledge is inevitable... but it's also different from what this article is about. Tacit knowledge is the kind of knowledge (and skill!) that cannot be f

Tacit knowledge is inevitable... but it's also different from what this article is about. Tacit knowledge is the kind of knowledge (and skill!) that cannot be fully explained or taught in words—think mechanical skills you can only pick up through physical practice or very context-specific expertise you only get through experience. In programming good taste is crucial and entirely tacit: we can try to distill taste into words and write books about it, but it will never be enough. Tacit knowledge is not the same as explicit knowledge that just happens to not be documented. Tacit knowledge is not documented because it is undocumentable. You cannot avoid tacit knowledge. Human language fundamentally cannot express all the knowledge that experts develop through years of practice and experience. Even if an expert can express their experience, there are simply ideas and skills that people cannot learn exclusively from words. This is an important distinction. Trying to eliminate undocumented explicit knowledge is useful for teams. Trying to eliminate tacit knowledge is disastrous. I've seen attempts to make tacit knowledge legible to management—it is one of the most direct ways to sabotage your own experts because experts fundamentally cannot make all their knowledge explicit. Expertise is tacit knowledge and tacit knowledge is expertise. People generally understand that design by committee or design by regulation has awful results; one of the main reasons for this is that committees require decisions to be fully explicit and explainable—which hamstrings creative problem-solving and pushes for poor design decisions. Exclusively explicit processes get you lowest-common-denominator results: not the lowest common denominator of the expertise present, but the lowest common denominator of what people can explicitly communicate and understand, which is even lower! If you don't let experts be experts, you won't get expert-level work.

fidotron 3y ago
This. This article confuses tacit knowledge for the phenomenon most eloquently described by a former (superb) sysadmin colleague “documentation is the enemy of job security”.
- pheres 3y ago
  One should not look to be unreplaceable, but rather the opposite. Only when you become replaceable at a certain task (or able to delegate it) you can progress to do new (more interesting) things.
  - geodel 3y ago
    It could go either way. Some places will like and promote one to do different/more interesting things. Other places will eliminate the position with k, thx, bye! Moreover usually people have only so much energy to be always concerned with job / career growth/ changes etc. They are fine their little pieces of knowledge that make them useful at work in a unique way.
    - oceanplexian 3y ago
      If your company has a habit of firing people that share information, or punish employees who work smarter instead of harder, it’s a sign that they might be out of touch with value creation and will eventually be punished by the market. Being fired from such a place might actually be a blessing, not a curse.
- justin_oaks 3y ago
  Is this true, though? After seeing enough bone-headed management decisions, I wonder if the bosses even know or care about documentation when making decisions to fire someone. From my point of view, the lack of documentation might be a reason to fire someone.
peteradio 3y ago
Was just arguing with my father whether the difficulty in transplanting TSMC was due to tacit knowledge or missing documentation. I argued that an automated process failing to run would inherently be missing documentation but I'm curious to see what kind of tacit information would be involved here.
nonrandomstring 3y ago
Documentation makes the world go round. Definitely appropriate for high-risk must-deliver projects, but not in all contexts. Is it maybe a business fantasy that we can simply juice the talent of a team and bottle it? Where does this idea come from other than good engineering practice? If you work with smart, talented people in a fast moving world it seems, as you say, inevitable that you face the "problem" of tacit knowledge and manage it positively. In Designing Sound (and my lectures on sound design) I researched and wrote about this as "ineffable knowledge". My observation in arts and design through the 90s and early 00s was that we regularly get people who are x100 superstars but they're unable to articulate exactly how they do things and pass that knowledge to others (I was researching this from a UI/UX point of view). That is /talent/ and it's why we value people as more than just interchangeable cogs. Much process knowledge, and crucially problem-solving knowledge in any project will reside in the heads of individuals. (TBF I think the article is more about tacit cultural knowledge within the team as a whole) Sure if we lose key individuals the project is threatened, but on the other hand trying to get them to do a full knowledge dump, such that any replacement would be frictionless, is a fools errand. Treating "information" that way and ignoring people is one-dimensional thinking. So I don't think it's helpful to talk about tacit knowledge itself as "dangerous". What is more dangerous is composing mediocre teams only from replaceable workers and spending too much of their time on documenting as a safety net when they should be pushing forward. Meanwhile a danger is in treating valuable people so badly their continuity is not assured. Far fewer people fall under the proverbial bus than simply leave for better jobs. So we could take the view that over-cautious self-reporting and documenting is a sign of, and attempt to manage, low commitment business attitudes, poor wages, conditions and low job-security.
- pixl97 3y ago
  Documentation is one of those things that once there's a lot of it, it's its own product. First, you have to have someone constantly making sure it's up to date in the case of software. Really easy for it to go stale and just be wrong, and possibly actively harmful. But then telling someone "RTFM" has a different quality if you're talking about 50 pages or 50,000 pages. Cool, I'll get back to you in a month.
nullhole 3y ago
It's worth noting a tautology: tacit knowledge is undocmented because it is undocumentable. The key insight (which I agree with!) is that some kinds of knowledge are undocumentable. The challenge - at least for those charged with ensuring processes are documented - is deciding what is and what isn't documentable.
- betenoire 3y ago
  I have a lot of habits because of lessons I've learned. There are 100 ways to do it, I chose 1. Documenting the 99 is unreasonable, but that's tacit knowledge, isn't it? I will document why I didn't choose the most obvious one, though.
- tikhonj 3y ago
  It's a definition. I suppose definitions are tautologies by definition though :) And yeah, the real insight is not the definition but the fact that tacit knowledge is inevitable and important, and that trying to eliminate it will be actively counterproductive.
jessriedel 3y ago
Thanks, this is the exactly correct critique of the article. I'll just add that the author did not seriously grapple with the fact that leaving at least some explicit knowledge undocumented is inevitable. There is no sharp divide between "explicit knowledge every programmer should know" and "explicit knowledge that only makes sense in our organization". Documenting is costly to produce, and documentation is costly to maintain even after it is produced. (If nothing else, more documentation makes it harder to search.) This holds even if it's simultaneously true that large organizations usually document insufficiently because of laziness and bad incentives.
- er4hn 3y ago
  I agree that some is inevitable, but as that amount increases it will provide a drag in what the organization is able to accomplish. This becomes even worse as an organization is remote locally and globally spread out.
  - k_process 3y ago
    The challenges with the post's suggestion are (1) it accounts only for the time of the person needing information, not the one providing it, and (2) a specific question takes a small time to answer, but pre-documenting everything that is needed to answer any question takes orders of magnitude more time. Some of this is outlined in Snowden's (not that Snowden) Laws of Knowledge Management. http://www.gurteen.com/gurteen/gurteen.nsf/id/km-seven-principles http://www.gurteen.com/gurteen/gurteen.nsf/id/km-seven-princ... These two are especially relevant: 2. We only know what we know when we need to know it. 7. We always know more than we can say, and we always say more than we can write down.
chongli 3y ago
I think almost all knowledge is tacit knowledge. Even stuff that would seem obviously explicit like mathematics and language. In my experience tutoring high school students, I’ve encountered plenty of students who struggle with spelling, grammar, or mathematical mistakes and no matter how many times I correct them it doesn’t stick. Then I see other students who pick these things up so quickly without correction. Mathematics, especially, seems to have a strong tacit component. If all you do is read the textbook you’re going to bomb the exam. You need to practice a lot to develop your mathematical intuition.
- panarky 3y ago
  >> Tacit knowledge is not documented because it is undocumentable > students who struggle with spelling The ability to spell correctly is not tacit knowledge. >> Tacit knowledge is not documented because it is undocumentable Spelling is thoroughly documented.
  - mnsc 3y ago
    It is thoroughly documented, yet alot of people make spelling mistakes and we sometime make up animals to mock people who spell stuff "as it sounds in their heads". And then there's people who are good spellers, whomst wouldn't blink twice or use Google when asked to spell "accommodate" or "nausaeus". Do you think that these mega spellers can document _how_ they spell correctly all the time? Or do you think they have some good intuition/tacit knowledge? English is my second language so I tried to make some funny mistakes above due to the topic, but there might also be some unwanted/undocumented ones thrown in for good measure.
    - aleph_minus_one 3y ago
      > Do you think that these mega spellers can document _how_ they spell correctly all the time? At least in my native language, I can typically explain the rules why a word is spelled the way it is. The simple reason for this is that I am interested in the structure of languages. The typical reason that I experience among other people why they spell words wrong is that they are less interested in this topic.
CrypticShift 3y ago
This is so meta. Here we are trying to verbalize our tacit knowledge of "tacit knowledge" :) While I agree with the distinction you made (Last paragraph is excellent), there is this grey area : Because in the digital era almost everything is recorded (oral meetings/written chitchat...), there is a huge quantity of information that is neither in the "Guts" nor in the "docs". To join others in this thread, I guess LLMs will play a big part in making this grey area usefull and in a certain way bridge the gap (partially) between what is tacit and what is formally documented.
- Lutger 3y ago
  Exactly. However I think its a bit more complicated than a grey area. There is just no hard distinction between tacit and explicit knowledge, even though the distinction is useful. It is not that any tacit bit of knowledge eludes articulation, it is the whole that is problematic. Nothing is truly undocumentable, but its like the specs for microsoft word: its just so much you already need to be an expert to navigate the whole of it. Thats the distinction and LLM of course of very good at processing vast amounts of bits and pieces.
ekidd 3y ago
> very context-specific expertise you only get through experience. When giving advice, I'm often tempted to point to a given situation, and to explain what rule of thumb I used to solve it. But then I would see a junior programmer applying the rule of thumb in a completely different situation and I'd want to say, "No, that will end in tears!" And I realized that a lot of my rules of thumb were really dumb when taken out of context. As far as I can tell, a huge part of expertise is having a large set of mappings like: - In contexts like "C1", use rule of thumb "R1" or "R8". It's very tempting to talk about R1, R2, etc. You can write blog posts like, "How R2 will transform your team!" But a lot of real expertise comes from being able to say, "Oh, this is a combination of C7 and C78, so I should consider rules of thumb R7, R49 and R102. Ah, wait, no, R49 is just gross here." So I've learned that, when mentoring, I need to spend a lot more time talking about the context before proposing any rules of thumb.
- enasterosophes 3y ago
  This is very insightful, thanks. I'm currently in the middle ground where I still want mentoring from the technical leads on some things, but I also have enough seniority to be mentoring new hires. You've articulated the way I think our tech leads are often not great at explaining themselves, while also giving me something to think about when passing on information to newbies.
- smilliken 3y ago
  My rule of thumb is to hesitate before giving advice if I can't come up with an anecdotal story to support it. Stories are more useful than generalizations because they come with the context, and can be dismissed in different contexts. At risk of presently breaking this rule of thumb, I'll give an example. When I notice code being structured with class inheritance, I'll point out a part of the codebase where I did similar, and how poorly it turned out, which someone can observe for themself. It started out nice, as it usually does, but devolved over time as complexity accumulated in parent classes to handle scenarios in child classes.
- AnimalMuppet 3y ago
  Um, can I retract about the last 15 years of my advice?
vishnugupta 3y ago
+1. This[1] article that was on HN front a few months ago and I could only agree with it. I've seen teams trying to micro-document their way to inefficiency. Forcing doers to document every darned small thing is the surest way to get them to lose interest. By now I've realised that regardless of documentation it takes around 6 months for a s/w engineer to be fully productive at a new company and start creating value. [1] https://commoncog.com/tacit-knowledge-is-a-real-thing/ https://commoncog.com/tacit-knowledge-is-a-real-thing/
vakkermans 3y ago
Well put! Donald Schön’s work on this topic is really enlightening. It’s not just that there’s knowledge in the heads of people that can’t be linguistically expressed well, but also that expression of it requires interaction with a specific situation. This is also represents a massive gap for AI systems to become actual in-the-world problems solvers.

fidotron 3y ago

This. This article confuses tacit knowledge for the phenomenon most eloquently described by a former (superb) sysadmin colleague “documentation is the enemy of job security”.

pheres 3y ago
One should not look to be unreplaceable, but rather the opposite. Only when you become replaceable at a certain task (or able to delegate it) you can progress to do new (more interesting) things.
- geodel 3y ago
  It could go either way. Some places will like and promote one to do different/more interesting things. Other places will eliminate the position with k, thx, bye! Moreover usually people have only so much energy to be always concerned with job / career growth/ changes etc. They are fine their little pieces of knowledge that make them useful at work in a unique way.
  - oceanplexian 3y ago
    If your company has a habit of firing people that share information, or punish employees who work smarter instead of harder, it’s a sign that they might be out of touch with value creation and will eventually be punished by the market. Being fired from such a place might actually be a blessing, not a curse.
justin_oaks 3y ago
Is this true, though? After seeing enough bone-headed management decisions, I wonder if the bosses even know or care about documentation when making decisions to fire someone. From my point of view, the lack of documentation might be a reason to fire someone.

peteradio 3y ago

Was just arguing with my father whether the difficulty in transplanting TSMC was due to tacit knowledge or missing documentation. I argued that an automated process failing to run would inherently be missing documentation but I'm curious to see what kind of tacit information would be involved here.

nonrandomstring 3y ago

Documentation makes the world go round. Definitely appropriate for high-risk must-deliver projects, but not in all contexts. Is it maybe a business fantasy that we can simply juice the talent of a team and bottle it? Where does this idea come from other than good engineering practice? If you work with smart, talented people in a fast moving world it seems, as you say, inevitable that you face the "problem" of tacit knowledge and manage it positively. In Designing Sound (and my lectures on sound design) I researched and wrote about this as "ineffable knowledge". My observation in arts and design through the 90s and early 00s was that we regularly get people who are x100 superstars but they're unable to articulate exactly how they do things and pass that knowledge to others (I was researching this from a UI/UX point of view). That is /talent/ and it's why we value people as more than just interchangeable cogs. Much process knowledge, and crucially problem-solving knowledge in any project will reside in the heads of individuals. (TBF I think the article is more about tacit cultural knowledge within the team as a whole) Sure if we lose key individuals the project is threatened, but on the other hand trying to get them to do a full knowledge dump, such that any replacement would be frictionless, is a fools errand. Treating "information" that way and ignoring people is one-dimensional thinking. So I don't think it's helpful to talk about tacit knowledge itself as "dangerous". What is more dangerous is composing mediocre teams only from replaceable workers and spending too much of their time on documenting as a safety net when they should be pushing forward. Meanwhile a danger is in treating valuable people so badly their continuity is not assured. Far fewer people fall under the proverbial bus than simply leave for better jobs. So we could take the view that over-cautious self-reporting and documenting is a sign of, and attempt to manage, low commitment business attitudes, poor wages, conditions and low job-security.

pixl97 3y ago
Documentation is one of those things that once there's a lot of it, it's its own product. First, you have to have someone constantly making sure it's up to date in the case of software. Really easy for it to go stale and just be wrong, and possibly actively harmful. But then telling someone "RTFM" has a different quality if you're talking about 50 pages or 50,000 pages. Cool, I'll get back to you in a month.

nullhole 3y ago

It's worth noting a tautology: tacit knowledge is undocmented because it is undocumentable. The key insight (which I agree with!) is that some kinds of knowledge are undocumentable. The challenge - at least for those charged with ensuring processes are documented - is deciding what is and what isn't documentable.

betenoire 3y ago
I have a lot of habits because of lessons I've learned. There are 100 ways to do it, I chose 1. Documenting the 99 is unreasonable, but that's tacit knowledge, isn't it? I will document why I didn't choose the most obvious one, though.
tikhonj 3y ago
It's a definition. I suppose definitions are tautologies by definition though :) And yeah, the real insight is not the definition but the fact that tacit knowledge is inevitable and important, and that trying to eliminate it will be actively counterproductive.

jessriedel 3y ago

Thanks, this is the exactly correct critique of the article. I'll just add that the author did not seriously grapple with the fact that leaving at least some explicit knowledge undocumented is inevitable. There is no sharp divide between "explicit knowledge every programmer should know" and "explicit knowledge that only makes sense in our organization". Documenting is costly to produce, and documentation is costly to maintain even after it is produced. (If nothing else, more documentation makes it harder to search.) This holds even if it's simultaneously true that large organizations usually document insufficiently because of laziness and bad incentives.

er4hn 3y ago
I agree that some is inevitable, but as that amount increases it will provide a drag in what the organization is able to accomplish. This becomes even worse as an organization is remote locally and globally spread out.
- k_process 3y ago
  The challenges with the post's suggestion are (1) it accounts only for the time of the person needing information, not the one providing it, and (2) a specific question takes a small time to answer, but pre-documenting everything that is needed to answer any question takes orders of magnitude more time. Some of this is outlined in Snowden's (not that Snowden) Laws of Knowledge Management. http://www.gurteen.com/gurteen/gurteen.nsf/id/km-seven-principles http://www.gurteen.com/gurteen/gurteen.nsf/id/km-seven-princ... These two are especially relevant: 2. We only know what we know when we need to know it. 7. We always know more than we can say, and we always say more than we can write down.

chongli 3y ago

I think almost all knowledge is tacit knowledge. Even stuff that would seem obviously explicit like mathematics and language. In my experience tutoring high school students, I’ve encountered plenty of students who struggle with spelling, grammar, or mathematical mistakes and no matter how many times I correct them it doesn’t stick. Then I see other students who pick these things up so quickly without correction. Mathematics, especially, seems to have a strong tacit component. If all you do is read the textbook you’re going to bomb the exam. You need to practice a lot to develop your mathematical intuition.

panarky 3y ago
>> Tacit knowledge is not documented because it is undocumentable > students who struggle with spelling The ability to spell correctly is not tacit knowledge. >> Tacit knowledge is not documented because it is undocumentable Spelling is thoroughly documented.
- mnsc 3y ago
  It is thoroughly documented, yet alot of people make spelling mistakes and we sometime make up animals to mock people who spell stuff "as it sounds in their heads". And then there's people who are good spellers, whomst wouldn't blink twice or use Google when asked to spell "accommodate" or "nausaeus". Do you think that these mega spellers can document _how_ they spell correctly all the time? Or do you think they have some good intuition/tacit knowledge? English is my second language so I tried to make some funny mistakes above due to the topic, but there might also be some unwanted/undocumented ones thrown in for good measure.
  - aleph_minus_one 3y ago
    > Do you think that these mega spellers can document _how_ they spell correctly all the time? At least in my native language, I can typically explain the rules why a word is spelled the way it is. The simple reason for this is that I am interested in the structure of languages. The typical reason that I experience among other people why they spell words wrong is that they are less interested in this topic.

CrypticShift 3y ago

This is so meta. Here we are trying to verbalize our tacit knowledge of "tacit knowledge" :) While I agree with the distinction you made (Last paragraph is excellent), there is this grey area : Because in the digital era almost everything is recorded (oral meetings/written chitchat...), there is a huge quantity of information that is neither in the "Guts" nor in the "docs". To join others in this thread, I guess LLMs will play a big part in making this grey area usefull and in a certain way bridge the gap (partially) between what is tacit and what is formally documented.

Lutger 3y ago
Exactly. However I think its a bit more complicated than a grey area. There is just no hard distinction between tacit and explicit knowledge, even though the distinction is useful. It is not that any tacit bit of knowledge eludes articulation, it is the whole that is problematic. Nothing is truly undocumentable, but its like the specs for microsoft word: its just so much you already need to be an expert to navigate the whole of it. Thats the distinction and LLM of course of very good at processing vast amounts of bits and pieces.

ekidd 3y ago

> very context-specific expertise you only get through experience. When giving advice, I'm often tempted to point to a given situation, and to explain what rule of thumb I used to solve it. But then I would see a junior programmer applying the rule of thumb in a completely different situation and I'd want to say, "No, that will end in tears!" And I realized that a lot of my rules of thumb were really dumb when taken out of context. As far as I can tell, a huge part of expertise is having a large set of mappings like: - In contexts like "C1", use rule of thumb "R1" or "R8". It's very tempting to talk about R1, R2, etc. You can write blog posts like, "How R2 will transform your team!" But a lot of real expertise comes from being able to say, "Oh, this is a combination of C7 and C78, so I should consider rules of thumb R7, R49 and R102. Ah, wait, no, R49 is just gross here." So I've learned that, when mentoring, I need to spend a lot more time talking about the context before proposing any rules of thumb.

enasterosophes 3y ago
This is very insightful, thanks. I'm currently in the middle ground where I still want mentoring from the technical leads on some things, but I also have enough seniority to be mentoring new hires. You've articulated the way I think our tech leads are often not great at explaining themselves, while also giving me something to think about when passing on information to newbies.
smilliken 3y ago
My rule of thumb is to hesitate before giving advice if I can't come up with an anecdotal story to support it. Stories are more useful than generalizations because they come with the context, and can be dismissed in different contexts. At risk of presently breaking this rule of thumb, I'll give an example. When I notice code being structured with class inheritance, I'll point out a part of the codebase where I did similar, and how poorly it turned out, which someone can observe for themself. It started out nice, as it usually does, but devolved over time as complexity accumulated in parent classes to handle scenarios in child classes.
AnimalMuppet 3y ago
Um, can I retract about the last 15 years of my advice?

vishnugupta 3y ago

+1. This[1] article that was on HN front a few months ago and I could only agree with it. I've seen teams trying to micro-document their way to inefficiency. Forcing doers to document every darned small thing is the surest way to get them to lose interest. By now I've realised that regardless of documentation it takes around 6 months for a s/w engineer to be fully productive at a new company and start creating value. [1] https://commoncog.com/tacit-knowledge-is-a-real-thing/ https://commoncog.com/tacit-knowledge-is-a-real-thing/

vakkermans 3y ago

Well put! Donald Schön’s work on this topic is really enlightening. It’s not just that there’s knowledge in the heads of people that can’t be linguistically expressed well, but also that expression of it requires interaction with a specific situation. This is also represents a massive gap for AI systems to become actual in-the-world problems solvers.