7 ms·

The diminishing half-life of knowledge

134 points by notaboredguy 3y ago

csneeky 3y ago
Darwin
lauriewired 3y ago
Personally, I find the 'knowledge half-life' problem to be easily solved with spaced repetition. I've used SuperMemo for a few years now, and have found it invaluable for retaining and revisiting old concepts, especially when switching between different tech stacks. The priority queue system of supermemo fixes the issues Anki and others have when collections grow to larger than 10K+ cards, where the daily load can often become unmanageable. I find 30 minutes a day of supermemo, supplemented by an hour of purposefully randomized review on weekends keeps even very old topics near the forefront of my mind. It's like having a well-maintained toolbox I can throw at different problems at any time. The slight daily effort is more than compensated by the retention gains I've noticed over time.
madaxe_again 3y ago
I think this is two phenomena being observed as one. While there’s certainly currently an accelerando going on in terms of knowledge growth and change in many fields, there is also bias built into us and our perception of the world - every human has a tendency to be surprised by the degree of change that occurs in their lifetime - we are raised with the world presented as-is, with a static set of truths, yet the only constant thing about the human world is change.
zer00eyz 3y ago
I often watch a pair of YouTubers who cover china, and they used to reside there. The one thing that they lamented often was a lack of maintenance, from historic buildings to lightbulbs in elevators. They tried to posit it as something more cultural, and I dont know if that is true or not for china. I do feel like the culture of tech has become one of maintenance not being part of what we do. When was the last time you saw someone get promoted for "cutting costs" or "making the code less complicated". When was the last time you sat down and read someone else's code and were impressed at how CLEAR and SIMPLE it was? 20 years ago we were all running on bare hardware. Now it's a hypervisor, with a VM, that has a docker container, that imports everything and a cat. We get tools like artifactory to make up for the fact that all of that might disappear. Top it of with zero ownership (infrastructure as a cost center and not a depreciable asset). It feels like a fuck shit stack of wet turds and were trying to play jenga with them, shuffling them back to the top and trying to get our next gig before the whole thing falls over and hopefully lands in someone else's lap. To make a point: do we need docker? No, but writing installable software is hard (depending on language and what you're doing). Docker doesn't fix the problem it just adds another layer in. The original service is the database. Yet we dont normally expose it because its security model remains rooted in the 19xx's. So we have layers of shims over it, ORM, JSON... pick your stack and "scalable" abstraction. The author mentions LLM's. The more I play with them the more I feel like this is going to be cool after engineers beat it into submission over the course of the next few years. So much opaque, and so little clarity on what is going on under the hood. It's no wonder people are having trouble coming to grips, it's a mess! If it were a battery break through it would take 10 years to turn it into a mass producible product, but because its software we throw money at it and let it dangle out on the web!!! (and I dont fear AGI) FTA: >> I don’t have a prescriptive solution for this. I wrote this text to start a discussion around a feeling I previously struggled with but didn’t know how to label. I do. We need to do better. We need to stop piling on more and strip back to clear and well understood. We need to value readable code over DRY, or Design patterns or what ever the next FOTM is. We need to laud people who make systems simpler, who reduces costs, who reshape and extend existing things rather than build new ones and pile more on top because it's "easy" or looks good on the resume. I am part of the problem too, and I need to stop. I need to stop reaching for that next shiny bit of tech, next framework, next library. Because acting like a schizophrenic ADHD child finding candy hasn't really served anyone.
- ysofunny 3y ago
  why fix anything if it's easier to add some other layer and kick the problems down the line? it's the economically rational thing to do... you woudln't want your kids to have a boring future gota leave some problems to them, heck cause some because we didn't know any better anyways
- sureglymop 3y ago
  Would you mind listing those youtubers which cover China? Sounds interesting. Also I'd like to kindly ask you not to use "ADHD child" in that manner because I think it stigmatizes it although I do understand the point you were trying to make there.
  - zer00eyz 3y ago
    Here is them covering the topic directly: https://www.youtube.com/watch?v=o9eXi3RL8q4 https://www.youtube.com/watch?v=o9eXi3RL8q4 AS for the ADHD thing, I get it, it's also a pretty accurate description of how I feel some days working in this industry. Its hard not to be a technological magpie collecting shiny rocks!
    - sshine 3y ago
      For context: both of these YouTubers were eventually denied stay in China and turned their channel into bashing China full time for a living. I really valued their insight and perspective (rural China by motorcycle, for example, is not a common perspective in the west), but eventually had to unsubscribe from their toxic bitterness.
      - jdewerd 3y ago
        Yeah, it's a shame they've been audience captured. At the beginning they leaned a bit to the rosy side, clearly glossing over visible negatives. Somewhere around when they left and were able to speak about good and bad but hadn't committed to a narrative was probably the point of peak value. Now they are almost comically anti-china. Ah well.
vaidhy 3y ago
IMHO, bullshit article. The author describes half-life as the time in which the knowledge is superseded or shown to be untrue. However, his actual example is around how he has forgotten math. It might be half-life of his retention, but not half-life of knowledge.
DeathArrow 3y ago
If you clicked the link hoping you will learn something about Gordon Freeman, you will be disappointed.
- arcanemachiner 3y ago
  If I came to HN and learned anything about Gordon Freeman, I would be disappointed.
eightnoteight 3y ago
while I agree that there is a half-life to certain type of knowledge but I think it would be overstating to reasonably apply to all knowledge. its very easy to retain a significant portion of the knowledge by building mental models. sure you will forget about the API surface of a technology, but you would surely remember the underlying knowledge models and you would surely apply it in many other contexts. like remembering the REST API inputs and output data types is also a knowledge whose half life is much shorter. but building mental models of those would stay for a long time. the point never is to remember and include REST API input and output data as part of your skill set, the point is to include their underlying knowledge. and you can't treat the API surface of a library as knowledge, its essentially a volatile memory and supposed to be cleared away
- eightnoteight 3y ago
  like who would care if you can bring certain type of knowledge's half life to 200 years. since you can't live anymore for that much time, you don't even need to think about solving that problem. just need to think about how to convert knowledge whose half life is less to the knowledge that has higher half life. (like converting API surfaces to mental models)
- jampekka 3y ago
  Exactly. It's weird for me that jobs are so strongly focused on some narrow technology. E.g. "React programmers" or "Javascript programmers" or "Python programmers". When you know the basic "underlying" model, switching from a framework or language to another is not a very big deal. Not much bigger deal than figuring out a new large codebase with a familiar framework/language. Sure, it may take a few weeks of learning new stuff and being unproductive (or even negatively productive) during this phase, but this is to be expected in any job. I don't think learning the more fundamental concepts is that hard but it does require some time (and interest) that is not immediately productive. Perhaps due to demands of being productive, as in churning code, all the time gets people (and the industry) to get stuck in such "local optimum".
  - heavenlyblue 3y ago
deleted 3y ago
[deleted]
nemoniac 3y ago
At the link below there's an interesting discussion of how an emphasis on competence-based education over knowledge-based education is not leading to an improvement in learning outcomes, quite the contrary. So despite the fact that knowledge may have a half-life, it may be even more important to acquire the skills to learn that knowledge. https://news.ycombinator.com/item?id=38590888 https://news.ycombinator.com/item?id=38590888
joshuaissac 3y ago
The article references the following IEEE Spectrum article: https://spectrum.ieee.org/an-engineering-career-only-a-young-persons-game https://spectrum.ieee.org/an-engineering-career-only-a-young... > Given a choice, many employers would rather hire a couple of inexperienced computer programmer and spend a few months training them than hiring (or retaining) a more experienced but expensive programmer. In the very next paragraph: > In addition, many employers aren’t interested in providing training to engineers or programmers who may start becoming obsolete, for fear of seeing them leave or be poached for employment elsewhere. [...] employers looking for experienced workers have fallen into the habit of poaching employees from their competitors as opposed to spending the resources to train from within their organization to meet a specific job skill. That directly contradicts the preceding paragraph, so I find it hard to trust the other claims that it makes.
- pylua 3y ago
  There does seem to be quite the inconsistency here. I feel like the truth lies somewhere near the truth of the no wants to work mantra which may simply be a refusal to raise wages. I wonder if the so called knowledge half life exists because everyone is working in very niche, specialized area these days. Those skills are simply less transferable to even similar jobs in the same field.
- jasode 3y ago
  >>, many employers would rather hire a couple of inexperienced computer programmer and spend a few months training them [...] In addition, many employers aren’t interested in providing training to engineers or programmers >That directly contradicts the preceding paragraph, The "many employers" can be 2 different subsets of employers and/or 2 different tech stack situations. Examples... Subset (1) FAANG or "tech" companies will train on specific in-house technology stacks for younger new hires. The "inexperienced" was in referencing "young". E.g. Apple hires fresh young college graduates that only did Scheme and Python in school but will train them on Objective-C and Swift so they can work on macOS and iOS. However, Apple typically doesn't hire older experienced COBOL programmers to re-train them in Swift. Subset (2) companies that don't train new hires (many non-tech companies where IT/dev is a cost center). They usually don't recruit from college campuses and prefer the job candidates to have existing skills that already match the job listing. E.g. a hospital IT's department has a job listing for a Java programmer to help maintain their billing system. The hospital is not interested in a candidate who's skillset is only Turbo Pascal and Macromedia ColdFusion and retraining them on Java.
roenxi 3y ago
As a counterpoint, it is impossible to discuss this topic without coming to some definition of "mastering" a tech stack. The concept itself seems suspicious. That is like saying "I mastered fashion". What, exactly, does it mean to master something that is in constant flux? How do we differentiate between a master and someone who read the documentation a few weeks ago? It is a mug's game trying to master a tech stack anyway. The clever thing to do is master problem domains. Then if you encounter the problem you can just solve it using old tools and be happy.
- joshuanapoli 3y ago
  Beyond some point, mastering "fundamentals" tends to look a bit like a barbershop pole. Engineering and computer science principles are in competition with each other. Within each tech stack, the balance of forces is different. It takes years of experience within a particular tech stack for an organization to find elegant and harmonious balance in techniques.
a_c 3y ago
Some angles for me to see this phenomenon - focus on the fundamentals rather than the particular brand of tools. HTTP, tcp, HTML, OS, CPU, filesystem, etc will almost certainly out last a language, framework and SaaS - See beyond the assumptions. Solutions are based on assumptions of current problem. Solutions come and go but the fundamental problem of, for example, go from a place to another, rarely change. Try to distill THE problem. - focus on outcome rather than action. Ask ourselves why such and such actions matter. Do we know why are we implementing a thing, or if a thing we implemented 6 months ago matter at all. All of these demands consideration beyond do you know "dewalts" or "bosch"
- marcosdumay 3y ago
  One really important issue with your first point is that everybody hiring is filtering by experience in SaaS, framework, and as a last resort, language. Nobody is searching for knowledge of the fundamentals. But anyway, my take is that the problem the article is describing is caused by the existence of way too many fundamentals that can't all be practiced.
  - 7thaccount 3y ago
    Both your comment and the parent you're replying to are awesome and match my experience in my area of electrical engineering (includes a lot of software, programming, database needs as well, so fairly relevant). There are those that have resumes tailored to a single particular thing that if hot right now, will have 1000 recruiters after them to run certain grid studies. I'm more of a fundamentalist (need to think of a better term)for my field where I can tell you the underlying math behind most of the studies performed, familiar with the pro/cons of a dozen different application softwares, can code, use SQL, Linux whatever. I'm also a people person, which is helpful as there is a lot of stakeholder interaction. If you understand the equations/theory and how all the technical junk works....there aren't many roles in my industry I can't get up to speed on in very little time, while having a global understanding for how it fits as a cog in the bigger machine. This isn't true of most in my industry unfortunately. Many many only know a single role, have experience with one tool, don't understand the math behind the tools, can't code or do data analysis... etc. I've found the broad/general experience to be very valuable, but it's harder to convey that to recruiters sometimes that are looking for "X". I sometimes have to tell them that what I have is highly transferable to "X" and that I have a bunch of other goodies their employer would be very interested in. Sometimes it works if they're communicating with the manager looking to fill the role and not HR. If I can actually talk to someone at the company.... usually not hard to arrange, then they've often offered to make a custom role as well. I know it always isn't the same for software shops or large companies with layers of beauracracy, but that's my experience.
epgui 3y ago
Obviously I don't have a solution to offer anyone, but this is one of my motivations for wanting to learn more about things like category theory, type theory, functional programming... It's not directly about getting skills that will land me a high paying job (via... marketable resume keywords?), it's more about trying to understand the fundamentals that no PL or framework trend, or social current, can obviate.
- starcraft2wol 3y ago
  I agree with the general idea of math and science. However the fields you mentioned are mostly useful for formalizing computer languages. If you want more concepts to use in actual programs, I think there are more practical areas to look in.
idkdotcom 3y ago
As the author highlights, this is not a new thing. What is definitely a more recent thing is that half-life is much shorter than before. Now it is fair to say that even the knowledge that's most staying power has at most a 2-3 year half life. My own solution to this problem has been to work in startups for 2-3 years and then move on rather than try to seek a well paid job at a large, prestigious company. Why? I started my career at one of these brand name, "everybody wants to get in" kind of companies that hadn't made institutional layoffs ever. When time came to do their first mass layoff, I saw people who had spent their entire careers at the company lose their jobs unemployable because thy had essentially become bureaucrats. Startups offer you the possibility of "on the job training" for the most recent technical stack. And precisely because there are too many people with golden handcuffs at the large companies, good startups are a bit easier to get into (not "easy", but "easier"). The downside is that 90% of startups fail, and you need to live with that. At the same time, if you happen to work at one that succeeds spectacularly, you won't have to worry about making a living until you die.
bawolff 3y ago
I think most "tech stack" knowledge is superficial. Knowing the tech stack is not the hard part of the job. The deep knowledge does not atrophy the same way. I think its kind of like the difference between being up to date with latest slang vs knowing how to read well.
jacquesm 3y ago
It's not so much the knowledge itself that has a half-life (unless it is front-end tech knowledge), but the ability to monetize knowledge. You used to be able to make a career out of some niche bit of knowledge but those days are over. You need to work hard just to stay current, in almost every field and that is as much a trend driven by technology as it is driven by the fact that there are so many people of working age now.
- zaptheimpaler 3y ago
  Credibly signalling that you do have some bit of knowledge is hard too. Its easy to learn a lot on your own today but the only widely valid signals are academic credentials or past experience.
- jandrewrogers 3y ago
  This is an excellent point. The frontier moves quickly, leaving a wake of commoditized skills behind it, but there used to be identifiable, well-paid skills that people chose precisely because the frontier moved slowly. While my instinct is to say that the frontier moves more quickly now, I have a suspicion that this less true than I think it is and what has really changed is that everything has a fast-moving frontier now. For example, the evolution and development of the Internet technology in the 1990s occurred at an incredible pace that is as fast as anything I've seen since, but back then you could switch to being e.g. an Oracle DBA if you wanted to avoid the chaos, and many people did. Those safe harbors have become rare in tech and the relative pay for them declines every year.
- zelphirkalt 3y ago
  This is why it is important to gain very good knowledge about the basics and "axioms" of whatever one works with. That way one can very quickly grasp "new" things, once it is needed. Without a solid grasp on the basics, one is bound to keep chasing the hype.
deleted 3y ago
[deleted]
aeonque 3y ago
In response to the spirit of the article and ignoring the specificity of content, a great way of retaining knowledge over time is Sean Whitelys Memletics courses. His frameworks and practices for studying, learning and retaining any subject matter are very practical. While his “Learn” tool can get a bit verbose, the concepts of reviewing notes in an irregular patterned method over time are extremely effective. I recommend using the Learn tool on a simple subject just to observe the effect the process has on your own experience of retaining knowledge over a few weeks, months and years. I don’t know where this guy Sean ended up - but if you get your hands on his materials, hold on to em! Hopefully that helps.
jandrewrogers 3y ago
It is important to recognize that even if knowledge becomes untrue because some assumption or fundamental has changed, knowing the history of these changes and why they occurred is still extraordinarily valuable knowledge. Too many software developers just know the "current thing" without knowing why it is the current thing and the specific issues that caused us to move on from the old thing. This ignorance of the past frequently encourages developers to reinvent an old thing poorly without understanding that not only is it not new but that we stopped doing it in many contexts for good and nuanced reasons. I think this sense of history is one of the most important aspects of "experience" when hiring. It is easy to train someone on an arbitrary tech stack but difficult to train someone on the history and path dependencies. Many developers are not interested in that history because it doesn't feel relevant to doing a job now. We tend to gain this sense of history by being in the industry long enough that we were part of that history.
- aleph_minus_one 3y ago
  > It is important to recognize that even if knowledge becomes untrue because some assumption or fundamental has changed, knowing the history of these changes and why they occurred is still extraordinarily valuable knowledge. [...] I think this sense of history is one of the most important aspects of "experience" when hiring. I can tell you that in my experience employers do not value this kind of knowledge a lot. Quite the opposite: quite some employers hate such employees who ask "too many inconvenient questions" instead of surfing the hype of the "current/next big thing".
  - dartos 3y ago
    While you’re not wrong, just bc employers don’t care about it during interviews doesn’t mean it’s not important. In my experience, startups with inexperienced technical leadership tend to be the “next big thing” (as far as tech stacks go) focused types of places.
  - JohnFen 3y ago
    In a sense, though, that doesn't matter. I mean, it would be nice, but it's not material. That knowledge is of value to the devs directly because it makes them better devs. That is something employers do actually value.
reactordev 3y ago
You get used to it. Just approach it every time as “learning a new stack” and those old muscle memories will come back automagically. Knowing how you got to Z from A is valuable knowledge too. Not just in changes to the old stack that is now new again, but to your own knowledge and journey. It’s ok to not be the smartest person in the room. It’s ok to admit that you need ramp up. What’s not ok is to go around claiming to be an expert and spout inaccurate facts so you’re already one step closer to being awesome in stack A again.
barrysteve 3y ago
When reading and writing was new, there was a real problem of authors becoming reactive and narrowing their writing to a reaction of what they read. The same concept can happen with coders. They don't have an understanding and purpose for code outside of the computer, and they code reactively. Therefore it is easy for them to forget. It is also worth noting that the culture and electronic lesuire has become very mentally consuming (if you allow it to be). It is possible to use electronic media so much that you forget code.
yetanother12345 3y ago
This is not specific to IT. It is a general trend. Also, it's not new; cf the saying "Jack of all trades, master of none". Also, the reason why we don't see Da Vinci types anymore. Enthropy is increasing. Things that used to be in one domain will in time split up into multiple, each having a higher level of detail / sophistication than before. Or, that one domain will die off, possibly being replaced by "something else". This can not be reversed - "you cannot unbreak the broken glass" (It seems that we need to be able to reverse our direction in spacetime to do that, and I believe that is considered a hard problem.) If you want deep knowledge you can't have broad knowledge. It follows from this that, eg AGI is deadborn; but then specialized AIs have huge potential. This should be the scary part, not the utopian know-it-all Mechanical Turk / HAL / Marvin. Speaking of the latter, we will have little use for an AGI anyway as such a thing will be way out of our league and we will not be able to use it, as D Adams famously postulated: "Here I am with a brain the size of a planet and they ask me to pick up a piece of paper. Call that job satisfaction? I don't."
- rightbyte 3y ago
  > "you cannot unbreak the broken glass" I really don't like this analogy. You can weld glass back together. Or melt the pieces down and reforge it.
  - jacquesm 3y ago
    Replace 'glass' with 'vase'. Analogies are always breaking down but break down faster if people don't just try to see them as analogies rather than the thing itself, the idea was to convey a point and you indicated that you got it perfectly well.
derrickpersson 3y ago
I've actually recognized this problem myself. I found that even though I took good notes, reviewed it somewhat consistently - when jumping back to something I used to be a 'master' in, my knowledge is still lacking. I'm building a learning focused note taking app for this very purpose - https://www.wonderpkm.com/ https://www.wonderpkm.com/ , would love to chat further with you about your learning approach!

134 points by notaboredguy 3y ago

Darwin

Personally, I find the 'knowledge half-life' problem to be easily solved with spaced repetition. I've used SuperMemo for a few years now, and have found it invaluable for retaining and revisiting old concepts, especially when switching between different tech stacks. The priority queue system of supermemo fixes the issues Anki and others have when collections grow to larger than 10K+ cards, where the daily load can often become unmanageable. I find 30 minutes a day of supermemo, supplemented by an hour of purposefully randomized review on weekends keeps even very old topics near the forefront of my mind. It's like having a well-maintained toolbox I can throw at different problems at any time. The slight daily effort is more than compensated by the retention gains I've noticed over time.

madaxe_again 3y ago

I think this is two phenomena being observed as one. While there’s certainly currently an accelerando going on in terms of knowledge growth and change in many fields, there is also bias built into us and our perception of the world - every human has a tendency to be surprised by the degree of change that occurs in their lifetime - we are raised with the world presented as-is, with a static set of truths, yet the only constant thing about the human world is change.

zer00eyz 3y ago

I often watch a pair of YouTubers who cover china, and they used to reside there. The one thing that they lamented often was a lack of maintenance, from historic buildings to lightbulbs in elevators. They tried to posit it as something more cultural, and I dont know if that is true or not for china. I do feel like the culture of tech has become one of maintenance not being part of what we do. When was the last time you saw someone get promoted for "cutting costs" or "making the code less complicated". When was the last time you sat down and read someone else's code and were impressed at how CLEAR and SIMPLE it was? 20 years ago we were all running on bare hardware. Now it's a hypervisor, with a VM, that has a docker container, that imports everything and a cat. We get tools like artifactory to make up for the fact that all of that might disappear. Top it of with zero ownership (infrastructure as a cost center and not a depreciable asset). It feels like a fuck shit stack of wet turds and were trying to play jenga with them, shuffling them back to the top and trying to get our next gig before the whole thing falls over and hopefully lands in someone else's lap. To make a point: do we need docker? No, but writing installable software is hard (depending on language and what you're doing). Docker doesn't fix the problem it just adds another layer in. The original service is the database. Yet we dont normally expose it because its security model remains rooted in the 19xx's. So we have layers of shims over it, ORM, JSON... pick your stack and "scalable" abstraction. The author mentions LLM's. The more I play with them the more I feel like this is going to be cool after engineers beat it into submission over the course of the next few years. So much opaque, and so little clarity on what is going on under the hood. It's no wonder people are having trouble coming to grips, it's a mess! If it were a battery break through it would take 10 years to turn it into a mass producible product, but because its software we throw money at it and let it dangle out on the web!!! (and I dont fear AGI) FTA: >> I don’t have a prescriptive solution for this. I wrote this text to start a discussion around a feeling I previously struggled with but didn’t know how to label. I do. We need to do better. We need to stop piling on more and strip back to clear and well understood. We need to value readable code over DRY, or Design patterns or what ever the next FOTM is. We need to laud people who make systems simpler, who reduces costs, who reshape and extend existing things rather than build new ones and pile more on top because it's "easy" or looks good on the resume. I am part of the problem too, and I need to stop. I need to stop reaching for that next shiny bit of tech, next framework, next library. Because acting like a schizophrenic ADHD child finding candy hasn't really served anyone.

ysofunny 3y ago
why fix anything if it's easier to add some other layer and kick the problems down the line? it's the economically rational thing to do... you woudln't want your kids to have a boring future gota leave some problems to them, heck cause some because we didn't know any better anyways
sureglymop 3y ago
Would you mind listing those youtubers which cover China? Sounds interesting. Also I'd like to kindly ask you not to use "ADHD child" in that manner because I think it stigmatizes it although I do understand the point you were trying to make there.
- zer00eyz 3y ago
  Here is them covering the topic directly: https://www.youtube.com/watch?v=o9eXi3RL8q4 https://www.youtube.com/watch?v=o9eXi3RL8q4 AS for the ADHD thing, I get it, it's also a pretty accurate description of how I feel some days working in this industry. Its hard not to be a technological magpie collecting shiny rocks!
  - sshine 3y ago
    For context: both of these YouTubers were eventually denied stay in China and turned their channel into bashing China full time for a living. I really valued their insight and perspective (rural China by motorcycle, for example, is not a common perspective in the west), but eventually had to unsubscribe from their toxic bitterness.
    - jdewerd 3y ago
      Yeah, it's a shame they've been audience captured. At the beginning they leaned a bit to the rosy side, clearly glossing over visible negatives. Somewhere around when they left and were able to speak about good and bad but hadn't committed to a narrative was probably the point of peak value. Now they are almost comically anti-china. Ah well.

vaidhy 3y ago

IMHO, bullshit article. The author describes half-life as the time in which the knowledge is superseded or shown to be untrue. However, his actual example is around how he has forgotten math. It might be half-life of his retention, but not half-life of knowledge.

DeathArrow 3y ago

If you clicked the link hoping you will learn something about Gordon Freeman, you will be disappointed.

arcanemachiner 3y ago
If I came to HN and learned anything about Gordon Freeman, I would be disappointed.

eightnoteight 3y ago

while I agree that there is a half-life to certain type of knowledge but I think it would be overstating to reasonably apply to all knowledge. its very easy to retain a significant portion of the knowledge by building mental models. sure you will forget about the API surface of a technology, but you would surely remember the underlying knowledge models and you would surely apply it in many other contexts. like remembering the REST API inputs and output data types is also a knowledge whose half life is much shorter. but building mental models of those would stay for a long time. the point never is to remember and include REST API input and output data as part of your skill set, the point is to include their underlying knowledge. and you can't treat the API surface of a library as knowledge, its essentially a volatile memory and supposed to be cleared away

eightnoteight 3y ago
like who would care if you can bring certain type of knowledge's half life to 200 years. since you can't live anymore for that much time, you don't even need to think about solving that problem. just need to think about how to convert knowledge whose half life is less to the knowledge that has higher half life. (like converting API surfaces to mental models)
jampekka 3y ago
Exactly. It's weird for me that jobs are so strongly focused on some narrow technology. E.g. "React programmers" or "Javascript programmers" or "Python programmers". When you know the basic "underlying" model, switching from a framework or language to another is not a very big deal. Not much bigger deal than figuring out a new large codebase with a familiar framework/language. Sure, it may take a few weeks of learning new stuff and being unproductive (or even negatively productive) during this phase, but this is to be expected in any job. I don't think learning the more fundamental concepts is that hard but it does require some time (and interest) that is not immediately productive. Perhaps due to demands of being productive, as in churning code, all the time gets people (and the industry) to get stuck in such "local optimum".
- heavenlyblue 3y ago

deleted 3y ago

[deleted]

nemoniac 3y ago

At the link below there's an interesting discussion of how an emphasis on competence-based education over knowledge-based education is not leading to an improvement in learning outcomes, quite the contrary. So despite the fact that knowledge may have a half-life, it may be even more important to acquire the skills to learn that knowledge. https://news.ycombinator.com/item?id=38590888 https://news.ycombinator.com/item?id=38590888

joshuaissac 3y ago

The article references the following IEEE Spectrum article: https://spectrum.ieee.org/an-engineering-career-only-a-young-persons-game https://spectrum.ieee.org/an-engineering-career-only-a-young... > Given a choice, many employers would rather hire a couple of inexperienced computer programmer and spend a few months training them than hiring (or retaining) a more experienced but expensive programmer. In the very next paragraph: > In addition, many employers aren’t interested in providing training to engineers or programmers who may start becoming obsolete, for fear of seeing them leave or be poached for employment elsewhere. [...] employers looking for experienced workers have fallen into the habit of poaching employees from their competitors as opposed to spending the resources to train from within their organization to meet a specific job skill. That directly contradicts the preceding paragraph, so I find it hard to trust the other claims that it makes.

pylua 3y ago
There does seem to be quite the inconsistency here. I feel like the truth lies somewhere near the truth of the no wants to work mantra which may simply be a refusal to raise wages. I wonder if the so called knowledge half life exists because everyone is working in very niche, specialized area these days. Those skills are simply less transferable to even similar jobs in the same field.
jasode 3y ago
>>, many employers would rather hire a couple of inexperienced computer programmer and spend a few months training them [...] In addition, many employers aren’t interested in providing training to engineers or programmers >That directly contradicts the preceding paragraph, The "many employers" can be 2 different subsets of employers and/or 2 different tech stack situations. Examples... Subset (1) FAANG or "tech" companies will train on specific in-house technology stacks for younger new hires. The "inexperienced" was in referencing "young". E.g. Apple hires fresh young college graduates that only did Scheme and Python in school but will train them on Objective-C and Swift so they can work on macOS and iOS. However, Apple typically doesn't hire older experienced COBOL programmers to re-train them in Swift. Subset (2) companies that don't train new hires (many non-tech companies where IT/dev is a cost center). They usually don't recruit from college campuses and prefer the job candidates to have existing skills that already match the job listing. E.g. a hospital IT's department has a job listing for a Java programmer to help maintain their billing system. The hospital is not interested in a candidate who's skillset is only Turbo Pascal and Macromedia ColdFusion and retraining them on Java.

roenxi 3y ago

As a counterpoint, it is impossible to discuss this topic without coming to some definition of "mastering" a tech stack. The concept itself seems suspicious. That is like saying "I mastered fashion". What, exactly, does it mean to master something that is in constant flux? How do we differentiate between a master and someone who read the documentation a few weeks ago? It is a mug's game trying to master a tech stack anyway. The clever thing to do is master problem domains. Then if you encounter the problem you can just solve it using old tools and be happy.

joshuanapoli 3y ago
Beyond some point, mastering "fundamentals" tends to look a bit like a barbershop pole. Engineering and computer science principles are in competition with each other. Within each tech stack, the balance of forces is different. It takes years of experience within a particular tech stack for an organization to find elegant and harmonious balance in techniques.

a_c 3y ago

Some angles for me to see this phenomenon - focus on the fundamentals rather than the particular brand of tools. HTTP, tcp, HTML, OS, CPU, filesystem, etc will almost certainly out last a language, framework and SaaS - See beyond the assumptions. Solutions are based on assumptions of current problem. Solutions come and go but the fundamental problem of, for example, go from a place to another, rarely change. Try to distill THE problem. - focus on outcome rather than action. Ask ourselves why such and such actions matter. Do we know why are we implementing a thing, or if a thing we implemented 6 months ago matter at all. All of these demands consideration beyond do you know "dewalts" or "bosch"

marcosdumay 3y ago
One really important issue with your first point is that everybody hiring is filtering by experience in SaaS, framework, and as a last resort, language. Nobody is searching for knowledge of the fundamentals. But anyway, my take is that the problem the article is describing is caused by the existence of way too many fundamentals that can't all be practiced.
- 7thaccount 3y ago
  Both your comment and the parent you're replying to are awesome and match my experience in my area of electrical engineering (includes a lot of software, programming, database needs as well, so fairly relevant). There are those that have resumes tailored to a single particular thing that if hot right now, will have 1000 recruiters after them to run certain grid studies. I'm more of a fundamentalist (need to think of a better term)for my field where I can tell you the underlying math behind most of the studies performed, familiar with the pro/cons of a dozen different application softwares, can code, use SQL, Linux whatever. I'm also a people person, which is helpful as there is a lot of stakeholder interaction. If you understand the equations/theory and how all the technical junk works....there aren't many roles in my industry I can't get up to speed on in very little time, while having a global understanding for how it fits as a cog in the bigger machine. This isn't true of most in my industry unfortunately. Many many only know a single role, have experience with one tool, don't understand the math behind the tools, can't code or do data analysis... etc. I've found the broad/general experience to be very valuable, but it's harder to convey that to recruiters sometimes that are looking for "X". I sometimes have to tell them that what I have is highly transferable to "X" and that I have a bunch of other goodies their employer would be very interested in. Sometimes it works if they're communicating with the manager looking to fill the role and not HR. If I can actually talk to someone at the company.... usually not hard to arrange, then they've often offered to make a custom role as well. I know it always isn't the same for software shops or large companies with layers of beauracracy, but that's my experience.

epgui 3y ago

Obviously I don't have a solution to offer anyone, but this is one of my motivations for wanting to learn more about things like category theory, type theory, functional programming... It's not directly about getting skills that will land me a high paying job (via... marketable resume keywords?), it's more about trying to understand the fundamentals that no PL or framework trend, or social current, can obviate.

starcraft2wol 3y ago
I agree with the general idea of math and science. However the fields you mentioned are mostly useful for formalizing computer languages. If you want more concepts to use in actual programs, I think there are more practical areas to look in.

idkdotcom 3y ago

As the author highlights, this is not a new thing. What is definitely a more recent thing is that half-life is much shorter than before. Now it is fair to say that even the knowledge that's most staying power has at most a 2-3 year half life. My own solution to this problem has been to work in startups for 2-3 years and then move on rather than try to seek a well paid job at a large, prestigious company. Why? I started my career at one of these brand name, "everybody wants to get in" kind of companies that hadn't made institutional layoffs ever. When time came to do their first mass layoff, I saw people who had spent their entire careers at the company lose their jobs unemployable because thy had essentially become bureaucrats. Startups offer you the possibility of "on the job training" for the most recent technical stack. And precisely because there are too many people with golden handcuffs at the large companies, good startups are a bit easier to get into (not "easy", but "easier"). The downside is that 90% of startups fail, and you need to live with that. At the same time, if you happen to work at one that succeeds spectacularly, you won't have to worry about making a living until you die.

bawolff 3y ago

I think most "tech stack" knowledge is superficial. Knowing the tech stack is not the hard part of the job. The deep knowledge does not atrophy the same way. I think its kind of like the difference between being up to date with latest slang vs knowing how to read well.

jacquesm 3y ago

It's not so much the knowledge itself that has a half-life (unless it is front-end tech knowledge), but the ability to monetize knowledge. You used to be able to make a career out of some niche bit of knowledge but those days are over. You need to work hard just to stay current, in almost every field and that is as much a trend driven by technology as it is driven by the fact that there are so many people of working age now.

zaptheimpaler 3y ago
Credibly signalling that you do have some bit of knowledge is hard too. Its easy to learn a lot on your own today but the only widely valid signals are academic credentials or past experience.
jandrewrogers 3y ago
This is an excellent point. The frontier moves quickly, leaving a wake of commoditized skills behind it, but there used to be identifiable, well-paid skills that people chose precisely because the frontier moved slowly. While my instinct is to say that the frontier moves more quickly now, I have a suspicion that this less true than I think it is and what has really changed is that everything has a fast-moving frontier now. For example, the evolution and development of the Internet technology in the 1990s occurred at an incredible pace that is as fast as anything I've seen since, but back then you could switch to being e.g. an Oracle DBA if you wanted to avoid the chaos, and many people did. Those safe harbors have become rare in tech and the relative pay for them declines every year.
zelphirkalt 3y ago
This is why it is important to gain very good knowledge about the basics and "axioms" of whatever one works with. That way one can very quickly grasp "new" things, once it is needed. Without a solid grasp on the basics, one is bound to keep chasing the hype.

deleted 3y ago

[deleted]

aeonque 3y ago

In response to the spirit of the article and ignoring the specificity of content, a great way of retaining knowledge over time is Sean Whitelys Memletics courses. His frameworks and practices for studying, learning and retaining any subject matter are very practical. While his “Learn” tool can get a bit verbose, the concepts of reviewing notes in an irregular patterned method over time are extremely effective. I recommend using the Learn tool on a simple subject just to observe the effect the process has on your own experience of retaining knowledge over a few weeks, months and years. I don’t know where this guy Sean ended up - but if you get your hands on his materials, hold on to em! Hopefully that helps.

jandrewrogers 3y ago

It is important to recognize that even if knowledge becomes untrue because some assumption or fundamental has changed, knowing the history of these changes and why they occurred is still extraordinarily valuable knowledge. Too many software developers just know the "current thing" without knowing why it is the current thing and the specific issues that caused us to move on from the old thing. This ignorance of the past frequently encourages developers to reinvent an old thing poorly without understanding that not only is it not new but that we stopped doing it in many contexts for good and nuanced reasons. I think this sense of history is one of the most important aspects of "experience" when hiring. It is easy to train someone on an arbitrary tech stack but difficult to train someone on the history and path dependencies. Many developers are not interested in that history because it doesn't feel relevant to doing a job now. We tend to gain this sense of history by being in the industry long enough that we were part of that history.

aleph_minus_one 3y ago
> It is important to recognize that even if knowledge becomes untrue because some assumption or fundamental has changed, knowing the history of these changes and why they occurred is still extraordinarily valuable knowledge. [...] I think this sense of history is one of the most important aspects of "experience" when hiring. I can tell you that in my experience employers do not value this kind of knowledge a lot. Quite the opposite: quite some employers hate such employees who ask "too many inconvenient questions" instead of surfing the hype of the "current/next big thing".
- dartos 3y ago
  While you’re not wrong, just bc employers don’t care about it during interviews doesn’t mean it’s not important. In my experience, startups with inexperienced technical leadership tend to be the “next big thing” (as far as tech stacks go) focused types of places.
- JohnFen 3y ago
  In a sense, though, that doesn't matter. I mean, it would be nice, but it's not material. That knowledge is of value to the devs directly because it makes them better devs. That is something employers do actually value.

reactordev 3y ago

You get used to it. Just approach it every time as “learning a new stack” and those old muscle memories will come back automagically. Knowing how you got to Z from A is valuable knowledge too. Not just in changes to the old stack that is now new again, but to your own knowledge and journey. It’s ok to not be the smartest person in the room. It’s ok to admit that you need ramp up. What’s not ok is to go around claiming to be an expert and spout inaccurate facts so you’re already one step closer to being awesome in stack A again.

barrysteve 3y ago

When reading and writing was new, there was a real problem of authors becoming reactive and narrowing their writing to a reaction of what they read. The same concept can happen with coders. They don't have an understanding and purpose for code outside of the computer, and they code reactively. Therefore it is easy for them to forget. It is also worth noting that the culture and electronic lesuire has become very mentally consuming (if you allow it to be). It is possible to use electronic media so much that you forget code.

yetanother12345 3y ago

This is not specific to IT. It is a general trend. Also, it's not new; cf the saying "Jack of all trades, master of none". Also, the reason why we don't see Da Vinci types anymore. Enthropy is increasing. Things that used to be in one domain will in time split up into multiple, each having a higher level of detail / sophistication than before. Or, that one domain will die off, possibly being replaced by "something else". This can not be reversed - "you cannot unbreak the broken glass" (It seems that we need to be able to reverse our direction in spacetime to do that, and I believe that is considered a hard problem.) If you want deep knowledge you can't have broad knowledge. It follows from this that, eg AGI is deadborn; but then specialized AIs have huge potential. This should be the scary part, not the utopian know-it-all Mechanical Turk / HAL / Marvin. Speaking of the latter, we will have little use for an AGI anyway as such a thing will be way out of our league and we will not be able to use it, as D Adams famously postulated: "Here I am with a brain the size of a planet and they ask me to pick up a piece of paper. Call that job satisfaction? I don't."

rightbyte 3y ago
> "you cannot unbreak the broken glass" I really don't like this analogy. You can weld glass back together. Or melt the pieces down and reforge it.
- jacquesm 3y ago
  Replace 'glass' with 'vase'. Analogies are always breaking down but break down faster if people don't just try to see them as analogies rather than the thing itself, the idea was to convey a point and you indicated that you got it perfectly well.

derrickpersson 3y ago

I've actually recognized this problem myself. I found that even though I took good notes, reviewed it somewhat consistently - when jumping back to something I used to be a 'master' in, my knowledge is still lacking. I'm building a learning focused note taking app for this very purpose - https://www.wonderpkm.com/ https://www.wonderpkm.com/ , would love to chat further with you about your learning approach!