13 ms·

I think there is one interesting angle to this problem. I am someone who grew up with the technology, as the levels of abstractions were being added. I am now

I think there is one interesting angle to this problem. I am someone who grew up with the technology, as the levels of abstractions were being added. I am now benefiting from all those accumulated decades of knowledge. As the IT / development world was changing, I had enormous privilege and comfort to learn the things at the pace they were happening. Being able to assimilate changes over long decades. Be a witness to the problems and logic behind all those new solutions. Understand how we come to have JavaScript and the browser mess we are in and so many other curious features of todays digital world. I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc. The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path. For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option. We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals.

3seashells 3y ago
Software archeology
beretguy 3y ago
There will always be hyper-nerds and just super smart people who will study all of the low level stuff. Granted, they will be vastly outnumbered by high level programmers.
- onetimeuse92304 3y ago
  That's what I mean by "precious few". I am continuously hiring new people, mostly senior devs and tech leads. Maybe one in a hundred has any understanding of what virtual memory is. Or why one process can't access memory of another process and, if you really want it, how to set it up. Two decades ago that was common knowledge. It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it. Aside maybe from a simple problem like DNS resolution error (and not even that -- most devs don't understand how DNS propagation works, for example). So if those problems happen, they are mostly deferring to me as a guy who can solve every problem. I see no clear path to achieving a goal of having someone else to learn to do the same.
  - lifeisstillgood 3y ago
    The thing is as someone who twenty plus years ago was able to write the webserver at socket level and so on, if you handed me a keyboard today I have forgotten all the context, the skills - they have pages out of my brain and now have redis and javascript and hadoop.
    - Brian_K_White 3y ago
      But you know those things exist and how things relate to each other and affect each other, or could affect each other, and you don't need to already have a mental text book in ram to solve a problem. I don't know you and yet I know this.
  - spaniard89277 3y ago
    Im from the new breed. I'd like to know all that but, where do I get the money? It's not like I can spend so much time tinkering without producing. If at least was able to purchease a home I wouldnt have this pressure. I guess in the US is different because people have higher salaries. You have to have the time/resources to dive that deep. Or maybe it's your job, line some of the folks in a local company that work in hard problems. But even they are not that well payed.
ChatGTP 3y ago
I feel similar, I was a Linux Sysadmin in the early days of Gentoo, studied C and Network programming, configured hardware and RAID arrays and did some courses like "From NAND to Tetris". Over the years I did a LOT of programming. When "the cloud" came out, I could easily understand the magic behind it all, so it made a lot of sense to me and it still does. I feel "computer literate", however, even a lot of solid but younger engineers I work with struggle with lower level concepts and just don't have the knowledge and experience with things like networking protocols to be able to solve hard problems.
EvanAnderson 3y ago
There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields. I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention". I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had to go out of my way to pick up a historical component to my IT education, whereas it seems like a historical foundation is part of traditional math, science, and engineering training. Vernor Vinge was right, though-- software (and hardware) archaeologist will be a real job.
- cjalmeida 3y ago
  I've heard that advanced math has a similar problem, that some subjects are quite deep that only a handful of people really understand the topic and passing the knowledge down to younger mathematicians is not a given.
  - alpaca128 3y ago
    I guess in IT it's the other way around, where you can start with advanced abstractions and frameworks without bothering what kind of rabbit hole is underneath (at least while it's going smooth), it gets deeper as you go closer to the hardware. While in maths you need to understand the basics to even know what's going on with more advanced areas. And this already applies to the fundamentals too, where in school it's expected you understand the stuff from previous years. At least that's my impression as a non-mathematician. And then there's the mix between the two in languages that rely heavily on powerful type systems.
    - oldsecondhand 3y ago
      No, you can do the same in math. It just means ignoring the proofs and just using the theorems as they are. However sometimes the application and proof requires similar ways of thinking, and you kind of miss out by skipping the proof. But the same can be occasionally true for software engineering.
  - 3y ago
enriquto 3y ago
> spending a decade working with a low level programming language before you jump into high level programming language is simply not an option. Nobody said that. But spending a week is definitely an option to all programmers, and many would benefit from it.
- cjalmeida 3y ago
  You can get ahead very far with a 16h free course on kernel development https://training.linuxfoundation.org/training/a-beginners-guide-to-linux-kernel-development-lfd103/ https://training.linuxfoundation.org/training/a-beginners-gu... A motivated person can run through it in a couple weeks while working a day job.
  - treadmill 3y ago
    Ugh, why do they want me to register to access a free course.
    - acheron 3y ago
      Data collection, of course.
- arnaudvalette 3y ago
  Do you have any ressources to share ? I am a self taught developer, I do stuff at my level (typescript, lisp), also because when you work alone, such high level languages allow you to express things nicely and fast. I kinda know how a cpu works, I understand what is going on when you perform some bare metal programming, but I never barely went down the rabbit hole and performed some low level stuff. I'm in the situation where I don't know what I should learn to be on par with graduates/undergraduates students. I am searching ressources about networking, operating systems, or anything that could enhance my proximity with low level stuff.
  - danielvaughn 3y ago
    I'm also self-taught. I've been doing it since (roughly) 2009, and like many others I started at the highest level of abstraction with front-end web dev. I've worked with some mid-level languages like Objective-C and Java, but like you I've never really dug deeper than that. I know this is tired and cliche at this point, but literally this week I sat down with ChatGPT and asked it to teach me how to write WAT (web assembly text format) so I could understand how memory is managed at a really low level (but not so low that I risk crashing my computer). It turned out to be super valuable. This is where AI shines for me - I can ask it any question that pops into my head, and also validate whether what it's telling me is true by running the code and seeing whether it works. It was amazing. If you're curious, I'm fine sharing a link to the chat: https://chat.openai.com/share/583bf23b-b43d-4566-956a-e92b6fb1750c https://chat.openai.com/share/583bf23b-b43d-4566-956a-e92b6f...
chilmers 3y ago
Unless you're two hundred years old, you joined the development of this technology in the middle. There was already decades of progress in electronics and telecommunications theory and engineering that everything you listed was built on top of. Your predecessors likely thought it was all newfangled abstractions that obscured more fundamental knowledge, and your successors will think the same about whatever world they grew up in vs. whatever one they leave.
- tgv 3y ago
  That's the kind of argumentation the article argues against. There's a middle ground. Or, if you prefer, there's an optimum somewhere, and right now, we're on a descending slope. To build mobile phones, we need a lot of tech built on tech build on tech. Dwarfs on giant's shoulders. And there's a place for people who don't understand the underlying tech. I've got a colleague who doesn't understand any of it, yet can do useful frontend work. But when you stand too high above the ground, the problems in the article become real, and that will get us stuck.
- Retric 3y ago
  Abstractions are only relevant when they are leaky. People alive today were flipping toggle switches to load individual machine codes into memory. Some of them literally knew what every single transistor in the machine was designed to do. Programmers at the time went from thinking these kit computers are toys to watching all the old ideas like virtual memory, cache, pipelining, networking, and multiple cores get reintroduced as the anemic transistor budget exploded. None of it was particularly novel, but the slow introduction of older ideas was a great way to learn all this stuff from the ground up.
- renegade-otter 3y ago
  I don't need to know assembly to write a web app - it's totally magic, and it's fine. However, if I am using an abstraction like an ORM without understanding what is happening with the database under the hood, or why it's getting slow all of a sudden, it is bound to bite me in the ass. I shudder to think of how much money and energy the world is burning every second just because of this one alone.
jl6 3y ago
One mitigation of the “mastering computers takes 30 years” thesis is that many of the abstractions and technologies that we spent those decades learning are now obsolete, and newly minted grads are blessed with learning materials and environments that make catching up easier than ever.
_kb 3y ago
I'd look at it in the opposite direction. Good abstractions are powerful when you understand what they build on. That recurses all the way down. Learning bottom up is a natural path, but there is still plenty of opportunity to explore top down. The difference is that top down discovery needs to be an intentional journey.
ksec 3y ago
>We, people who really understand the technology that the world runs on, are a slowly dying breed. And web development has the worst tech abstraction of all. There used to be a time where any Software Engineers ( i think the term wasn't even invented then ) or simply programmers would know at least a thing or two about Hardware. We now have so much abstractions we have people working in tech who have zero knowledge on either software or hardware, or any low level stuff. That is why not only do we need some open source or open standards ( I am not a zealot for everything open sources ), we also need to simplify everything we have today. To basically refractor everything we have learned from Hardware to Software.
- blacktonystark 3y ago
  To be fair, its hard for anyone to know anything about hardware these days unless its a company with a large on-prem footprint and it doesn't make financial or regulatory sense to put in the cloud or co-locate, which is a vanishingly small number. Hell I haven't been in a physical datacenter in 5 years and that's where I started my career many moons ago.
BiteCode_dev 3y ago
You can actually see this with new frontend devs. They know only full SPA frameworks, they have never seen a dump of an HTTP message, headers and verbs are abstract things to them. Hell, many of them don't know you can have fully functional websites with zero JS, including payment, video, login, etc. I started to write an HTMX tutorial (https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1 https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1) because I noticed a lot of young coders don't understand what to do with it. They read the tweets saying it's nice, but when they look at it, it makes no sense to them. It's really fun because I now remember how some senior coders looked at me, knowing nothing about compilation. I was struggling with Python packaging because before wheels, it required compiling a lot on linux, and it failed often. For them it was obvious: just install the headers, look you need the dev packages, wait, you don't have gcc? Nowadays I happily patch nginx source code and compile it manually, but it took a lot of work to learn a minuscule chunk of what all those guys knew by heart. Because they started with assembly.
- ilyt 3y ago
  Ask a full stack JS developer to make their app use a proxy and they will get nervous breakdown
- ksec 3y ago
  >You can actually see this with new frontend devs. They know only full SPA frameworks, ...... Around I guess 2017 or even 2016? I used to think this is some sort internet troll comment about people never seen a dump of an HTTP message, until...... >Because they started with assembly. It wasn't necessary starting with assembly or something low level. ( Although that certainly helped ) We have less entertainment, more time, and no Internet ( for most part ). Things also dont work all the time. And we have to spend time figuring it out ourselves. That is where all the knowledge comes from.
  - BiteCode_dev 3y ago
    Having to figure it out is not specific to a generation. My generation had magazines and forums. Then came out google and SO. Now chat gpt. The difference is where you start figuring things out. Before, you needed to figure things out at your level, because it was the immediate area of mystery. Now this level is generally solved, you need to figure things out at a different level: - filter out the mass of irrelevant information, the out dates one, and the one from spam - understand how all those complex abstractions interact. We have good resources on how each work individually, but the carthesian product of the monsters we build with them, we obviously don't - debug some things that don't work when the magic fails, way below your level, or behind a service, where nobody is looking The problem before was scarcity of info, lack of standardization and roughness of systems. Now it's abundance, opacity and too much sophistication. But everybody still have to figure things out. Just no the same challenges.
djur 3y ago
This is not consistent with my experience with younger devs. There are teenagers doing kernel-level hacking and there's also older people who have been comfortably driving RAD tools for decades. This is fine. Software development has been gradually specializing and it will continue to do so. The tools and resources available for people who want to pursue low-level engineering are better, cheaper, and more accessible than ever.
- worthless-trash 3y ago
  My $management have said that there are nearly no teenagers who will apply for kernel work. You got someone I can reach out to when the option comes up again ?
devjab 3y ago
If you’re managers and tech leads then why aren’t you making sure the people you’re supposed to lead obtain the understanding? We work with both high and low level tools where I work, and in my experience there isn’t necessarily a lack of understanding among younger developers. The CS courses still teach you a lot of the basics, and to actually use what you call “hype” tools efficiently you typically need a rather good understanding of how they work. I actually don’t think we have enough abstraction, especially not in the DevOps field. In my opinion we’ve overcomplicated the deployment procedure without making sure it was abstracted. We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I’m actually fine with infrastructure as code, but it needs to be templated to the point that our developers simply “order”resources based on whatever template they want to deploy so that they don’t have to keep up with the constant changes or learn how networking in enterprise organisations work. As far as the CPU stuff goes. Young developers do a lot of low level code. We write a lot of embedded software for our solar plants, and a lot of the time, they seem far more capable and “modern” in their approaches than the “old guard” exactly because they’ve been taught the same curriculum but also all the lesson learned by the “old guard” along the way. That being said, it’s not really necessary for a lot of developers to keep an active knowledge of how an x86 CPU works, because it’s very unlike that they’ll have to work with it. It will often be far more useful to know how various types of ARM processors work, as that’s something a lot of us actively work with.
- Viliam1234 3y ago
  > We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I am afraid that many businesses prefer the vision where the software developers do the operations too, and the company can fire the operations department. Consider how much money you could save... before it all falls apart. Short term optimization is how managers get their bonuses. The trick is to move on to the next project before the old one falls apart. Then it becomes someone else's problem.
coldtea 3y ago
>The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path. The worst problem, which TFA alludes to, is that those "layers" are not really needed, they are leaky kludges upon kludges to make some lower layer unsuitable for later needs more palatable or able to handle some unforeseen use case that's against its design. Other stuff is just added as ways to sell new enterprise tooling, support and consulting. Seeing the layers and technology stacks being added over time can give the impression of watching some organic evolution happen, when it's often merely accumulation, rough patching, and corporate attempt to push its technologies/NIH.
- onetimeuse92304 3y ago
  Yes. And we know this process from legacy code. Rather than fix the problem, it is usually easier to just write some new code to manage the old problem. We tack on new layers, modules, abstractions, workarounds all because at the time it seemed to be easier than refactoring the cause of the problem. But it only works so far and at some point the application becomes unmaintainable mess and the effort to rewrite it from scratch starts. Only we can't rewrite the world after it has become so unmaintainable that nobody can figure out how to change a detail of some layer inbetween the program and the transistor. Heck, we are already stuck trying to fix the bad design of IPv4.
eddyg 3y ago
Agreed! It reminds me of: https://github.com/alex/what-happens-when https://github.com/alex/what-happens-when and how many of today’s CS-degree holders would barely understand any of it. As someone who has also “grown up with all the technology”, I’ve learned and experienced all that. But as a percentage of “software engineers”, there’s fewer and fewer that do every day.
- CapstanRoller 3y ago
  Unfortunately, knowing all this won't help you get a job when the market demands an army of React devs and little else. Under our current system, knowledge that cannot be monetized is useless. Worse than useless, in fact, because time spent learning low-level concepts is time that could be spent learning marketable skills. The industry is effectively paying us all to forget. Sure, learning the USB stack and differential signaling is fun, but unless my job involves implementing custom USB devices from scratch then it's pointless trivia that won't pay the bills.
ixaxaar 3y ago
Hey I've struggled with this notion as well. But then I sometimes compare this with crypto - or how a bunch of kids simulated the entire financial system and made the same mistakes made 100s of years ago, with the end effect of an new generation that taught itself finance. I suppose that is what will keep happening. At some point, someone will decide to re-write everything from scratch and re-discover lower layers of abstraction or challenge them.
mensetmanusman 3y ago
A sci-fi semi-solution is to hope that all the relevant information is embodied in these LLMs being built. A funny possible history is everyone forgetting what’s necessary and then having the relevant information they need accidentally blocked by an AI system they have no idea how to build or fix (maybe because of terms like master/slave going against elite values).
WanderPanda 3y ago
I used to think that the historical development of a subject can not possibly be the best way to learn a new topic because it is so path dependent and random. But with experience I have seen that there are very few instances where educators mixed up the historical timeline of a subject and made it more easily absorbable. It still baffles me that this is even the case for abstract fields like math
bheadmaster 3y ago
> people who really understand the technology that the world runs on, are a slowly dying breed Not completely. Curious people will always pop up, it's just that they will take as much time as you did to learn everything. I like to think of myself as one of the curious people. Ever since I was a kid, I had a huge desire to understand how the hell does this magic box called a computer work. Decades later, as a software engineer, no matter how much I learn, no matter how deep I go, there's always this voice in my head going "but WHY? WHY are things the way they are?". I did start top-down instead of bottom-up, starting from the high-level languages running on modern operating systems, going more and more into the (professionally unnecessary) low-level through pure curiosity, but I still believe I will get to the bottom one day.
yodsanklai 3y ago
> For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option As a counterpoint, I'd say that it's also chance to build on top of these abstractions without having your brain cluttered with how they're implemented. That's also how science grows. And don't underestimate the new players. They're perfectly capable to understand the low level details if they have to, and they have much more resource available to learn too.
FpUser 3y ago
I am exactly the same, coming from making my own add on boards for early PCs and programming in machine codes and then following tech as it developed. Yes it is hard to find young persons with the same level of understanding but they do exists.
scottmcdot 3y ago
Where do we draw the line though? Surely those proficient in low level programming languages don't then need to become proficient in building motherboards from scratch?
- delecti 3y ago
  I think that, at the very least, there's a huge benefit in having a bit of understanding one abstraction down from everything you interact with. Knowing what an abstraction is abstracting, and why, goes a long way in coexisting with that abstraction. A low-level programmer doesn't need to actually build motherboards from scratch, but they'd probably benefit a lot from a bit of understanding about processor caches, registers, and bus latencies.
maxrecursion 3y ago
"But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals." I just hope when that time comes I'm well compensated for studying erudite texts on low level computer science while chilling on my phone instead of browsing social media or playing games. But honestly, I enjoy reading and learning about ABIs, C programming, network protocols, etc... more than mindless scrolling anyway.
- bloomingeek 3y ago
  Although I worked with computers most of my career, it wasn't in the tech sphere at all. The article meant a lot to me because of trying to wade through tons of websites for info about signing up for SS and Medicare. These sites were not set up for non-computer tech people. Like you, I enjoy learning techie stuff, but most of my friends don't. Because, IMO, most websites are written for younger people, they feel almost helpless.
slibhb 3y ago
This is a non-problem. If there's a shortage of people working at X level of abstraction (where X is possibly very low) then salaries for those with that knowledge will go up and more people will seek it out. Free markets don't solve every problem but they solve this one.
vsareto 3y ago
>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals. I'm gonna guess this same message, in different words, has been repeated often throughout history for as long as people had the ability to reflect on their life and the next generation. So really, what's different this time? What's the difference between forgotten technology that nobody needs and an unpublished author who's works were forgotten? I think the answer is none. We just think it's more important because this is the industry we care about. We're actually really good at reinventing the wheel in this industry, so even if some software is forgotten, you can probably bet it'll be reinvented or reverse-engineered if it's really needed. While we should preserve technology knowledge (because it's pretty cheap to do so), in terms of the economy, sometimes things are a stepping stone and should be forgotten.
- deleted 3y ago
  [deleted]
tester756 3y ago
>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. There will still be people at e.g faang/semiconductor companies that do this stuff day2day and even push advances in those areas and have already created solid training materials for younger employees If you're worried that this knowledge will disappear, then feel free to write it up in form of tutorials.
deleted 3y ago
[deleted]
snakey 3y ago
Excellent point. Giving my perspective as a relative newbie (~5 years) to the industry with no comp. sci. background. It was around the year 3-4 mark that I decided to knuckle down and try to improve my fundamentals (data structures, algorithms, memory models, concurrency, CPU architecture and some network fundamentals) by reading popular papers and literature and writing all of my personal projects in C and C++. I’m about two years into this study and while it’s been immensely rewarding, I’ve found it to be a huge undertaking while juggling life and a full-time job. What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work (still Python and JS). I would love to be able to put these concepts into practice for many hours of my working day but it’s difficult to move from general web-stack development to more performance-oriented development (embedded, low-latency, OS, etc.). My guess is that this is one of the reasons we have ended up in this situation. You can get away without knowing the fundamentals (a good sign of progress?) and that if you really do want to pursue these areas that promote building this kind of knowledge as part of your career, the barrier to entry is quite high and the positions are fewer than say a decade or two ago. I find it a shame because in my eyes, these areas are the most interesting and exciting areas of programming. It’s an art.
- lolinder 3y ago
  > What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work I think this is because of the effect OP is describing: no matter how much time you spent studying, the odds that you learned about the specific thing that will make the difference in your work this week are slim because the technology is so broad and deep. Odds are that what you choose to intentionally study is irrelevant to your work. You've picked a slice to learn about, but those who were working on it as it got layered know it all. Once you have that kind of knowledge, the odds of being able to explain some unexpected behavior approach 100%, but acquiring that knowledge took decades. I've found a method that does work pretty well is to dig really deep into the topics that come up in your actual work. Instead of a random sample of fundamentals, find the parts of your job that feel like magic and explain that magic. Skipping over V8 and straight to the CPU might not be useful for a JavaScript dev very often, but a deep understanding of V8 is relevant more often than you might think.
AnimalMuppet 3y ago
I think this is unduly pessimistic. Take low-level kernel work, for instance. Between Linux, Microsoft, and Apple, there are lots of people who understand it. It's just a smaller proportion than used to understand it. The absolute number of people has probably grown. But we don't need every programmer to understand low-level kernel stuff, precisely because the kernel abstraction is good enough at hiding that stuff. Most of us don't need to care (whereas 40 years ago, many more did need to care). Personally, I myself never want to care about low-level kernel things. When I'm trying to write a program to do something, if I can write it without having to worry about low-level things, that's a win.
corndoge 3y ago
This sounds plausible but I don't think it's correct. The body of knowledge you refer to is large but not that large. I think by the time a teenager with a persistent interest in computers is 25 or so they can accumulate a similar level of expertise, and without no lifing it.
notacoward 3y ago
I believe that the first lesson plan for any subject should be to follow its historical progression. That's not the be-all and end-all but, whether you're talking about computer science or physics or economics, it's a good starting point because it's one proven way to deal with the field's complexity. You just have to be clear that the simpler models from older times are pedagogical tools, not current understanding.
supportengineer 3y ago
I have that knowledge but don’t know how to monetize it. Kids are coming out of school with 5 YOE and making double my salary with almost 30 YOE. And they ONLY know how to do React.
steveBK123 3y ago
Completely agreed. For example, I am NOT a cloud expert, and barely a novice, by any measure. When dealing with various CloudOps/Cloud Architect/etc types, without fail, the good ones are also complete Linux gurus who would be at home architecting on-prem infrastructure as well. Unfortunately there are a lot of "experts" who are only comfortable working in abstraction and so as soon as we have an issue that scratches below the surface, it turns into an all-hands-on-deck mayday situation, looping in other teams/experts to try and save them. I've also seen horrific implementations by abstraction enjoyers. Imagine a continuous integration environment of the following - k8s where one pod simply runs a forever shell script that is basically "git checkout master;git pull; sleep 300;". Then the other pod actually runs the application on top of the shared storage that the first pod writes to. This script, unsurprisingly, fails silently and hangs frequently. To debug this you need to go through some cloud auth portal and click through some web UI to then open a console in your browser (which doesn't support copy/paste). The idiot version of doing this on-prem would be a single small VM that has a cron running every 5 minutes, which invokes your deployment script, with cron configured to email on failure. This would be simpler, more reliable (not relying on a forever script), and more supportable (it emails on fail!). And it uses like 30 year old tech that just fkn works.
detourdog 3y ago
I totally relate to your point of view and experience. I'm so happy to ignore all the software related to the attention economy. What I find left over is the computing environment that I dreamt of when I was 12. I marvel that it took a whole generation of man hours to get to this point. I think we are real inflection point between an analog world and a digital world. I also think that the Bronze Age never went away and analog techniques will just develop into craftsmanship. I haven't posted a link to these Autonetics parts in while so I will do so again. What I love about Autonetics is that they were solving the inertial guidance problem in whatever computing form they could get a hold of. I'm working on a proper website but the images linked here represent the inertial guidance problem being solved with discrete components arranged in circuits and later those circuits integrated in silicon. https://en.wikipedia.org/wiki/Autonetics https://en.wikipedia.org/wiki/Autonetics https://www.icloud.com/sharedalbum/#B0YG4TcsmGWIVSf https://www.icloud.com/sharedalbum/#B0YG4TcsmGWIVSf Some Nixdorf 820 photos https://www.computerhistory.org/revolution/memory-storage/8/264/1116 https://www.computerhistory.org/revolution/memory-storage/8/... https://www.icloud.com/sharedalbum/#B0Y59UlCqNHCjx https://www.icloud.com/sharedalbum/#B0Y59UlCqNHCjx
yterdy 3y ago
Older workers are supposed to pass along this knowledge to younger workers in an expedited fashion. Acquisition of the understanding of increasingly complex systems must be optimized to operational needs. One of the many, many gripes Millennials have with Baby Boomers and Gen-Xers is that they jealously guard institutional knowledge in the name of job security, for years, and then just... retire. If you're lucky, your company will shell out massive consultation fees to have them come back and drip-feed what they know to younger personnel. It's not just the natural order of things that such knowledge passes out of this world. It's one of many collective and affirmative decisions by people born before 1980 to take as much with them as possible when they go. I would be skeptical if it were just one thing, but after affordable college, affordable housing, affordable healthcare, affordable retirement, and a viable biosphere, it's become a pattern. That you seem to understand this on some level and don't care to try to mke a difference is even worse.
lolinder 3y ago
An important thing to remember is that not only did you get to watch the layers be added, you also have decades of experience. In today's field, where we double in number every N years, that's enough time to accumulate enough knowledge to feel like a demigod—to have more than 20 years of experience is extremely rare in the industry right now. That won't always be the case: eventually we'll stop growing exponentially and we'll find an equilibrium, and eventually the people with 30 years of experience will stop being a vanishingly small minority. While that may not give us the kind of breadth that your generation has, I would expect many of them to be able to master the specific slice of the tech tree that they actually build on top of. 30 years is plenty of time to learn the everything from the browser down to the CPU, or the HTTP framework down, or the database down, and probably enough to learn two such slices. Part of why it seems impossible now is just because most developers not only don't have the historical context, they haven't yet had any time.
- convolvatron 3y ago
  its more than just time. the sad part about this evoluation is that younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago. i think whats required to move forward isn't just some dusty studying of these physics, but a reawakening to an understanding that these are just systems with tradeoffs, and we can choose to make other systems just as easily. thats the only way the sandpile collapses and we get .. for example .. secure operating systems and network protocols. or systems that were designed to exploit the massive amount of concurrency available in modern machines.
  - lolinder 3y ago
    > younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago. This is simply not my experience of young people or of being young. There's a reason why radical movements are led and populated largely by the young—to the extent they err, it's on the side of "let's tear it all down and start over".
NohatCoder 3y ago
For the last 10 years you have been adding stuff that did nothing but increase the complexity. Now you wonder why new people can't keep up with the complexity.
tight-ship 3y ago
> perseverance to really understand all those things by diving into obscure, historical manuals. You say you were lucky to be there in the beginning and that young people don't have this advantage. Then you conflate your own luck with perseverance.
- onetimeuse92304 3y ago
  I am not conflating anything. For me it was my day to day work for decades. Some new technology happened and I had to learn it to be on top of my game. Trying to learn all this in 1/10th of the time to start your career is what requires perseverance.
  - tight-ship 3y ago
    May I quote? > We, *people who really understand the technology* that the world runs on, are a slowly dying breed Thus one has to conclude that young people, not having perseverance, do not really understand technology (and lacking such basic skill, never will).
bbarn 3y ago
I think you may be confusing what you see with the cause. For example: You and I HAD to be willing to go through that, and we were fascinated by it. There are still plenty of great FE and BE young devs who see something and instantly ask "How does this work" and keep going down. The difference is, most people in employment don't HAVE to do this like all people in our early career did. If those shortcuts to getting paid existed in our early careers, many of us would have taken the money and not cared. It's not an experience or a "You had to be there" thing, it's simple curiosity and motivation that sets people apart in this regard.
- sgarland 3y ago
  Precisely this. Some people have an innate intellectual curiosity. Some do not. I do not enjoy working with the latter.
konschubert 3y ago
Which is fine, it's specialisation. Of course, it's bad if the western world fully outsources certain components of the stack. But not everybody needs to understand the full stack. Maybe you don't know how the silicon crystals are made and that is okay.
nonrandomstring 3y ago
> We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals Some people like to say this is a perennial problem that every generation bemoans. I get why that might seem intuitive, and for the longest time I've hoped they're right and I'm merely getting old. Sadly they are wrong. Long term historical patterns of empire collapse follow this over-reach and under-education cycle that leads to a catastrophic rebuild capability gap that is triggered at some point. Thomas Thwaite's "The toaster project" [0] is a wonderful commentary on this. I cited it in a talk I did about education in world that will no longer pay for teaching kids how to use chips and breadboards in skinflint universities that prefer to teach them Microsoft Excel or use some crappy simulators of everything [1]. When it comes to cybersecurity we can only obtain defence in depth if we have knowledge in depth, and there are ever fewer of us around who understand how computers actually work. It's more than a little unnerving to meet CISOs and folks with high-flying titles who have no idea why using a phone app to control a Fortune 500 company infra might not be a good idea. [0] http://www.thetoasterproject.org/ http://www.thetoasterproject.org/ [1] https://www.youtube.com/watch?v=AxoSskalOrM https://www.youtube.com/watch?v=AxoSskalOrM
lr4444lr 3y ago
I suspect this happens in all technical professions as things advance: you learn first principles and low level implementations in a formal training program/degree, but the demands of the industry to be productive require that you use and trust abstractions. Even my doctors don't know some things about various drugs they prescribe me, because I can afford to spend the time reading the literature that only pertains to me. I'm sure their biochemistry background means they could definitely outperform me if they wanted to do a deep dive, but they know the important parts, and they know how my condition stacks up in the compendium of total knowledge and clinical experience they have. I don't think anything important is lost if they cannot at the drop of a hat tell me the half life and mechanism of action if I grill them on it.
dimmke 3y ago
This is the perfect articulation of something I think about all the time. It is so much easier if you were in it while it was happening. I've been trying to write a blog post about this, and how to address the problem specifically for front end development, where it's absolutely ludicrous. It's a very serious problem, especially when you think about security. You have all these people clamoring to get cloud certifications so they can work in IT but its so complex. A certification isn't enough. There just aren't enough people with enough knowledge and practical experience to fill all the needs that companies have.
deleted 3y ago
[deleted]
SubjectToChange 3y ago
I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc. If by "understand" you mean "have a passing familiarity with" then.. sure, but if you mean "my understanding is reflective of reality" then there's absolutely no way. We, people who really understand the technology that the world runs on, are a slowly dying breed. There are more people doing "low level" programming right now than were doing it a few decades ago. The only knowledge that is fading is knowledge that's applicable to legacy and/or retro systems.

3seashells 3y ago

Software archeology

beretguy 3y ago

There will always be hyper-nerds and just super smart people who will study all of the low level stuff. Granted, they will be vastly outnumbered by high level programmers.

onetimeuse92304 3y ago
That's what I mean by "precious few". I am continuously hiring new people, mostly senior devs and tech leads. Maybe one in a hundred has any understanding of what virtual memory is. Or why one process can't access memory of another process and, if you really want it, how to set it up. Two decades ago that was common knowledge. It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it. Aside maybe from a simple problem like DNS resolution error (and not even that -- most devs don't understand how DNS propagation works, for example). So if those problems happen, they are mostly deferring to me as a guy who can solve every problem. I see no clear path to achieving a goal of having someone else to learn to do the same.
- lifeisstillgood 3y ago
  The thing is as someone who twenty plus years ago was able to write the webserver at socket level and so on, if you handed me a keyboard today I have forgotten all the context, the skills - they have pages out of my brain and now have redis and javascript and hadoop.
  - Brian_K_White 3y ago
    But you know those things exist and how things relate to each other and affect each other, or could affect each other, and you don't need to already have a mental text book in ram to solve a problem. I don't know you and yet I know this.
- spaniard89277 3y ago
  Im from the new breed. I'd like to know all that but, where do I get the money? It's not like I can spend so much time tinkering without producing. If at least was able to purchease a home I wouldnt have this pressure. I guess in the US is different because people have higher salaries. You have to have the time/resources to dive that deep. Or maybe it's your job, line some of the folks in a local company that work in hard problems. But even they are not that well payed.

ChatGTP 3y ago

I feel similar, I was a Linux Sysadmin in the early days of Gentoo, studied C and Network programming, configured hardware and RAID arrays and did some courses like "From NAND to Tetris". Over the years I did a LOT of programming. When "the cloud" came out, I could easily understand the magic behind it all, so it made a lot of sense to me and it still does. I feel "computer literate", however, even a lot of solid but younger engineers I work with struggle with lower level concepts and just don't have the knowledge and experience with things like networking protocols to be able to solve hard problems.

EvanAnderson 3y ago

There will be an increasingly important history component in comp sci., software engineering, infosec, and associated fields. I think this is a place where the IT field has done a bad job and, as a result, we've had unbelievable amounts of "wheel reinvention". I'd love to hear from somebody with experience in scientific or other disciplines and IT who could contrast their fields with ours. I only know that I have had to go out of my way to pick up a historical component to my IT education, whereas it seems like a historical foundation is part of traditional math, science, and engineering training. Vernor Vinge was right, though-- software (and hardware) archaeologist will be a real job.

cjalmeida 3y ago
I've heard that advanced math has a similar problem, that some subjects are quite deep that only a handful of people really understand the topic and passing the knowledge down to younger mathematicians is not a given.
- alpaca128 3y ago
  I guess in IT it's the other way around, where you can start with advanced abstractions and frameworks without bothering what kind of rabbit hole is underneath (at least while it's going smooth), it gets deeper as you go closer to the hardware. While in maths you need to understand the basics to even know what's going on with more advanced areas. And this already applies to the fundamentals too, where in school it's expected you understand the stuff from previous years. At least that's my impression as a non-mathematician. And then there's the mix between the two in languages that rely heavily on powerful type systems.
  - oldsecondhand 3y ago
    No, you can do the same in math. It just means ignoring the proofs and just using the theorems as they are. However sometimes the application and proof requires similar ways of thinking, and you kind of miss out by skipping the proof. But the same can be occasionally true for software engineering.
- 3y ago

enriquto 3y ago

> spending a decade working with a low level programming language before you jump into high level programming language is simply not an option. Nobody said that. But spending a week is definitely an option to all programmers, and many would benefit from it.

cjalmeida 3y ago
You can get ahead very far with a 16h free course on kernel development https://training.linuxfoundation.org/training/a-beginners-guide-to-linux-kernel-development-lfd103/ https://training.linuxfoundation.org/training/a-beginners-gu... A motivated person can run through it in a couple weeks while working a day job.
- treadmill 3y ago
  Ugh, why do they want me to register to access a free course.
  - acheron 3y ago
    Data collection, of course.
arnaudvalette 3y ago
Do you have any ressources to share ? I am a self taught developer, I do stuff at my level (typescript, lisp), also because when you work alone, such high level languages allow you to express things nicely and fast. I kinda know how a cpu works, I understand what is going on when you perform some bare metal programming, but I never barely went down the rabbit hole and performed some low level stuff. I'm in the situation where I don't know what I should learn to be on par with graduates/undergraduates students. I am searching ressources about networking, operating systems, or anything that could enhance my proximity with low level stuff.
- danielvaughn 3y ago
  I'm also self-taught. I've been doing it since (roughly) 2009, and like many others I started at the highest level of abstraction with front-end web dev. I've worked with some mid-level languages like Objective-C and Java, but like you I've never really dug deeper than that. I know this is tired and cliche at this point, but literally this week I sat down with ChatGPT and asked it to teach me how to write WAT (web assembly text format) so I could understand how memory is managed at a really low level (but not so low that I risk crashing my computer). It turned out to be super valuable. This is where AI shines for me - I can ask it any question that pops into my head, and also validate whether what it's telling me is true by running the code and seeing whether it works. It was amazing. If you're curious, I'm fine sharing a link to the chat: https://chat.openai.com/share/583bf23b-b43d-4566-956a-e92b6fb1750c https://chat.openai.com/share/583bf23b-b43d-4566-956a-e92b6f...

chilmers 3y ago

Unless you're two hundred years old, you joined the development of this technology in the middle. There was already decades of progress in electronics and telecommunications theory and engineering that everything you listed was built on top of. Your predecessors likely thought it was all newfangled abstractions that obscured more fundamental knowledge, and your successors will think the same about whatever world they grew up in vs. whatever one they leave.

tgv 3y ago
That's the kind of argumentation the article argues against. There's a middle ground. Or, if you prefer, there's an optimum somewhere, and right now, we're on a descending slope. To build mobile phones, we need a lot of tech built on tech build on tech. Dwarfs on giant's shoulders. And there's a place for people who don't understand the underlying tech. I've got a colleague who doesn't understand any of it, yet can do useful frontend work. But when you stand too high above the ground, the problems in the article become real, and that will get us stuck.
Retric 3y ago
Abstractions are only relevant when they are leaky. People alive today were flipping toggle switches to load individual machine codes into memory. Some of them literally knew what every single transistor in the machine was designed to do. Programmers at the time went from thinking these kit computers are toys to watching all the old ideas like virtual memory, cache, pipelining, networking, and multiple cores get reintroduced as the anemic transistor budget exploded. None of it was particularly novel, but the slow introduction of older ideas was a great way to learn all this stuff from the ground up.
renegade-otter 3y ago
I don't need to know assembly to write a web app - it's totally magic, and it's fine. However, if I am using an abstraction like an ORM without understanding what is happening with the database under the hood, or why it's getting slow all of a sudden, it is bound to bite me in the ass. I shudder to think of how much money and energy the world is burning every second just because of this one alone.

jl6 3y ago

One mitigation of the “mastering computers takes 30 years” thesis is that many of the abstractions and technologies that we spent those decades learning are now obsolete, and newly minted grads are blessed with learning materials and environments that make catching up easier than ever.

_kb 3y ago

I'd look at it in the opposite direction. Good abstractions are powerful when you understand what they build on. That recurses all the way down. Learning bottom up is a natural path, but there is still plenty of opportunity to explore top down. The difference is that top down discovery needs to be an intentional journey.

ksec 3y ago

>We, people who really understand the technology that the world runs on, are a slowly dying breed. And web development has the worst tech abstraction of all. There used to be a time where any Software Engineers ( i think the term wasn't even invented then ) or simply programmers would know at least a thing or two about Hardware. We now have so much abstractions we have people working in tech who have zero knowledge on either software or hardware, or any low level stuff. That is why not only do we need some open source or open standards ( I am not a zealot for everything open sources ), we also need to simplify everything we have today. To basically refractor everything we have learned from Hardware to Software.

blacktonystark 3y ago
To be fair, its hard for anyone to know anything about hardware these days unless its a company with a large on-prem footprint and it doesn't make financial or regulatory sense to put in the cloud or co-locate, which is a vanishingly small number. Hell I haven't been in a physical datacenter in 5 years and that's where I started my career many moons ago.

BiteCode_dev 3y ago

You can actually see this with new frontend devs. They know only full SPA frameworks, they have never seen a dump of an HTTP message, headers and verbs are abstract things to them. Hell, many of them don't know you can have fully functional websites with zero JS, including payment, video, login, etc. I started to write an HTMX tutorial (https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1 https://www.bitecode.dev/p/a-little-taste-of-htmx-part-1) because I noticed a lot of young coders don't understand what to do with it. They read the tweets saying it's nice, but when they look at it, it makes no sense to them. It's really fun because I now remember how some senior coders looked at me, knowing nothing about compilation. I was struggling with Python packaging because before wheels, it required compiling a lot on linux, and it failed often. For them it was obvious: just install the headers, look you need the dev packages, wait, you don't have gcc? Nowadays I happily patch nginx source code and compile it manually, but it took a lot of work to learn a minuscule chunk of what all those guys knew by heart. Because they started with assembly.

ilyt 3y ago
Ask a full stack JS developer to make their app use a proxy and they will get nervous breakdown
ksec 3y ago
>You can actually see this with new frontend devs. They know only full SPA frameworks, ...... Around I guess 2017 or even 2016? I used to think this is some sort internet troll comment about people never seen a dump of an HTTP message, until...... >Because they started with assembly. It wasn't necessary starting with assembly or something low level. ( Although that certainly helped ) We have less entertainment, more time, and no Internet ( for most part ). Things also dont work all the time. And we have to spend time figuring it out ourselves. That is where all the knowledge comes from.
- BiteCode_dev 3y ago
  Having to figure it out is not specific to a generation. My generation had magazines and forums. Then came out google and SO. Now chat gpt. The difference is where you start figuring things out. Before, you needed to figure things out at your level, because it was the immediate area of mystery. Now this level is generally solved, you need to figure things out at a different level: - filter out the mass of irrelevant information, the out dates one, and the one from spam - understand how all those complex abstractions interact. We have good resources on how each work individually, but the carthesian product of the monsters we build with them, we obviously don't - debug some things that don't work when the magic fails, way below your level, or behind a service, where nobody is looking The problem before was scarcity of info, lack of standardization and roughness of systems. Now it's abundance, opacity and too much sophistication. But everybody still have to figure things out. Just no the same challenges.

djur 3y ago

This is not consistent with my experience with younger devs. There are teenagers doing kernel-level hacking and there's also older people who have been comfortably driving RAD tools for decades. This is fine. Software development has been gradually specializing and it will continue to do so. The tools and resources available for people who want to pursue low-level engineering are better, cheaper, and more accessible than ever.

worthless-trash 3y ago
My $management have said that there are nearly no teenagers who will apply for kernel work. You got someone I can reach out to when the option comes up again ?

devjab 3y ago

If you’re managers and tech leads then why aren’t you making sure the people you’re supposed to lead obtain the understanding? We work with both high and low level tools where I work, and in my experience there isn’t necessarily a lack of understanding among younger developers. The CS courses still teach you a lot of the basics, and to actually use what you call “hype” tools efficiently you typically need a rather good understanding of how they work. I actually don’t think we have enough abstraction, especially not in the DevOps field. In my opinion we’ve overcomplicated the deployment procedure without making sure it was abstracted. We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I’m actually fine with infrastructure as code, but it needs to be templated to the point that our developers simply “order”resources based on whatever template they want to deploy so that they don’t have to keep up with the constant changes or learn how networking in enterprise organisations work. As far as the CPU stuff goes. Young developers do a lot of low level code. We write a lot of embedded software for our solar plants, and a lot of the time, they seem far more capable and “modern” in their approaches than the “old guard” exactly because they’ve been taught the same curriculum but also all the lesson learned by the “old guard” along the way. That being said, it’s not really necessary for a lot of developers to keep an active knowledge of how an x86 CPU works, because it’s very unlike that they’ll have to work with it. It will often be far more useful to know how various types of ARM processors work, as that’s something a lot of us actively work with.

Viliam1234 3y ago
> We don’t need our software developers to know how our Cloud Networking and Virtual Networking works, we don’t need them to spin up resources and make various internal DNS and Firewall rules, we simply need them to focus on what they’re good at and then hand off the container to the operations department where people actually specialise in that knowledge. I am afraid that many businesses prefer the vision where the software developers do the operations too, and the company can fire the operations department. Consider how much money you could save... before it all falls apart. Short term optimization is how managers get their bonuses. The trick is to move on to the next project before the old one falls apart. Then it becomes someone else's problem.

coldtea 3y ago

>The issue, as with any automation, is that new players on the scene (younger devs, devops, etc.) simply have no chance to learn the same things and go trough the same path. The worst problem, which TFA alludes to, is that those "layers" are not really needed, they are leaky kludges upon kludges to make some lower layer unsuitable for later needs more palatable or able to handle some unforeseen use case that's against its design. Other stuff is just added as ways to sell new enterprise tooling, support and consulting. Seeing the layers and technology stacks being added over time can give the impression of watching some organic evolution happen, when it's often merely accumulation, rough patching, and corporate attempt to push its technologies/NIH.

onetimeuse92304 3y ago
Yes. And we know this process from legacy code. Rather than fix the problem, it is usually easier to just write some new code to manage the old problem. We tack on new layers, modules, abstractions, workarounds all because at the time it seemed to be easier than refactoring the cause of the problem. But it only works so far and at some point the application becomes unmaintainable mess and the effort to rewrite it from scratch starts. Only we can't rewrite the world after it has become so unmaintainable that nobody can figure out how to change a detail of some layer inbetween the program and the transistor. Heck, we are already stuck trying to fix the bad design of IPv4.

eddyg 3y ago

Agreed! It reminds me of: https://github.com/alex/what-happens-when https://github.com/alex/what-happens-when and how many of today’s CS-degree holders would barely understand any of it. As someone who has also “grown up with all the technology”, I’ve learned and experienced all that. But as a percentage of “software engineers”, there’s fewer and fewer that do every day.

CapstanRoller 3y ago
Unfortunately, knowing all this won't help you get a job when the market demands an army of React devs and little else. Under our current system, knowledge that cannot be monetized is useless. Worse than useless, in fact, because time spent learning low-level concepts is time that could be spent learning marketable skills. The industry is effectively paying us all to forget. Sure, learning the USB stack and differential signaling is fun, but unless my job involves implementing custom USB devices from scratch then it's pointless trivia that won't pay the bills.

ixaxaar 3y ago

Hey I've struggled with this notion as well. But then I sometimes compare this with crypto - or how a bunch of kids simulated the entire financial system and made the same mistakes made 100s of years ago, with the end effect of an new generation that taught itself finance. I suppose that is what will keep happening. At some point, someone will decide to re-write everything from scratch and re-discover lower layers of abstraction or challenge them.

mensetmanusman 3y ago

A sci-fi semi-solution is to hope that all the relevant information is embodied in these LLMs being built. A funny possible history is everyone forgetting what’s necessary and then having the relevant information they need accidentally blocked by an AI system they have no idea how to build or fix (maybe because of terms like master/slave going against elite values).

WanderPanda 3y ago

I used to think that the historical development of a subject can not possibly be the best way to learn a new topic because it is so path dependent and random. But with experience I have seen that there are very few instances where educators mixed up the historical timeline of a subject and made it more easily absorbable. It still baffles me that this is even the case for abstract fields like math

bheadmaster 3y ago

> people who really understand the technology that the world runs on, are a slowly dying breed Not completely. Curious people will always pop up, it's just that they will take as much time as you did to learn everything. I like to think of myself as one of the curious people. Ever since I was a kid, I had a huge desire to understand how the hell does this magic box called a computer work. Decades later, as a software engineer, no matter how much I learn, no matter how deep I go, there's always this voice in my head going "but WHY? WHY are things the way they are?". I did start top-down instead of bottom-up, starting from the high-level languages running on modern operating systems, going more and more into the (professionally unnecessary) low-level through pure curiosity, but I still believe I will get to the bottom one day.

yodsanklai 3y ago

> For them, spending a decade working with a low level programming language before you jump into high level programming language is simply not an option As a counterpoint, I'd say that it's also chance to build on top of these abstractions without having your brain cluttered with how they're implemented. That's also how science grows. And don't underestimate the new players. They're perfectly capable to understand the low level details if they have to, and they have much more resource available to learn too.

FpUser 3y ago

I am exactly the same, coming from making my own add on boards for early PCs and programming in machine codes and then following tech as it developed. Yes it is hard to find young persons with the same level of understanding but they do exists.

scottmcdot 3y ago

Where do we draw the line though? Surely those proficient in low level programming languages don't then need to become proficient in building motherboards from scratch?

delecti 3y ago
I think that, at the very least, there's a huge benefit in having a bit of understanding one abstraction down from everything you interact with. Knowing what an abstraction is abstracting, and why, goes a long way in coexisting with that abstraction. A low-level programmer doesn't need to actually build motherboards from scratch, but they'd probably benefit a lot from a bit of understanding about processor caches, registers, and bus latencies.

maxrecursion 3y ago

"But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals." I just hope when that time comes I'm well compensated for studying erudite texts on low level computer science while chilling on my phone instead of browsing social media or playing games. But honestly, I enjoy reading and learning about ABIs, C programming, network protocols, etc... more than mindless scrolling anyway.

bloomingeek 3y ago
Although I worked with computers most of my career, it wasn't in the tech sphere at all. The article meant a lot to me because of trying to wade through tons of websites for info about signing up for SS and Medicare. These sites were not set up for non-computer tech people. Like you, I enjoy learning techie stuff, but most of my friends don't. Because, IMO, most websites are written for younger people, they feel almost helpless.

slibhb 3y ago

This is a non-problem. If there's a shortage of people working at X level of abstraction (where X is possibly very low) then salaries for those with that knowledge will go up and more people will seek it out. Free markets don't solve every problem but they solve this one.

vsareto 3y ago

>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals. I'm gonna guess this same message, in different words, has been repeated often throughout history for as long as people had the ability to reflect on their life and the next generation. So really, what's different this time? What's the difference between forgotten technology that nobody needs and an unpublished author who's works were forgotten? I think the answer is none. We just think it's more important because this is the industry we care about. We're actually really good at reinventing the wheel in this industry, so even if some software is forgotten, you can probably bet it'll be reinvented or reverse-engineered if it's really needed. While we should preserve technology knowledge (because it's pretty cheap to do so), in terms of the economy, sometimes things are a stepping stone and should be forgotten.

deleted 3y ago
[deleted]

tester756 3y ago

>We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. There will still be people at e.g faang/semiconductor companies that do this stuff day2day and even push advances in those areas and have already created solid training materials for younger employees If you're worried that this knowledge will disappear, then feel free to write it up in form of tutorials.

deleted 3y ago

[deleted]

snakey 3y ago

Excellent point. Giving my perspective as a relative newbie (~5 years) to the industry with no comp. sci. background. It was around the year 3-4 mark that I decided to knuckle down and try to improve my fundamentals (data structures, algorithms, memory models, concurrency, CPU architecture and some network fundamentals) by reading popular papers and literature and writing all of my personal projects in C and C++. I’m about two years into this study and while it’s been immensely rewarding, I’ve found it to be a huge undertaking while juggling life and a full-time job. What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work (still Python and JS). I would love to be able to put these concepts into practice for many hours of my working day but it’s difficult to move from general web-stack development to more performance-oriented development (embedded, low-latency, OS, etc.). My guess is that this is one of the reasons we have ended up in this situation. You can get away without knowing the fundamentals (a good sign of progress?) and that if you really do want to pursue these areas that promote building this kind of knowledge as part of your career, the barrier to entry is quite high and the positions are fewer than say a decade or two ago. I find it a shame because in my eyes, these areas are the most interesting and exciting areas of programming. It’s an art.

lolinder 3y ago
> What I’ve also noticed is that, while I understand a lot more about what the CPU is doing, memory manipulation and how to write more efficient programs, I haven’t found it be particularly beneficial to my daily work I think this is because of the effect OP is describing: no matter how much time you spent studying, the odds that you learned about the specific thing that will make the difference in your work this week are slim because the technology is so broad and deep. Odds are that what you choose to intentionally study is irrelevant to your work. You've picked a slice to learn about, but those who were working on it as it got layered know it all. Once you have that kind of knowledge, the odds of being able to explain some unexpected behavior approach 100%, but acquiring that knowledge took decades. I've found a method that does work pretty well is to dig really deep into the topics that come up in your actual work. Instead of a random sample of fundamentals, find the parts of your job that feel like magic and explain that magic. Skipping over V8 and straight to the CPU might not be useful for a JavaScript dev very often, but a deep understanding of V8 is relevant more often than you might think.

AnimalMuppet 3y ago

I think this is unduly pessimistic. Take low-level kernel work, for instance. Between Linux, Microsoft, and Apple, there are lots of people who understand it. It's just a smaller proportion than used to understand it. The absolute number of people has probably grown. But we don't need every programmer to understand low-level kernel stuff, precisely because the kernel abstraction is good enough at hiding that stuff. Most of us don't need to care (whereas 40 years ago, many more did need to care). Personally, I myself never want to care about low-level kernel things. When I'm trying to write a program to do something, if I can write it without having to worry about low-level things, that's a win.

corndoge 3y ago

This sounds plausible but I don't think it's correct. The body of knowledge you refer to is large but not that large. I think by the time a teenager with a persistent interest in computers is 25 or so they can accumulate a similar level of expertise, and without no lifing it.

notacoward 3y ago

I believe that the first lesson plan for any subject should be to follow its historical progression. That's not the be-all and end-all but, whether you're talking about computer science or physics or economics, it's a good starting point because it's one proven way to deal with the field's complexity. You just have to be clear that the simpler models from older times are pedagogical tools, not current understanding.

supportengineer 3y ago

I have that knowledge but don’t know how to monetize it. Kids are coming out of school with 5 YOE and making double my salary with almost 30 YOE. And they ONLY know how to do React.

steveBK123 3y ago

Completely agreed. For example, I am NOT a cloud expert, and barely a novice, by any measure. When dealing with various CloudOps/Cloud Architect/etc types, without fail, the good ones are also complete Linux gurus who would be at home architecting on-prem infrastructure as well. Unfortunately there are a lot of "experts" who are only comfortable working in abstraction and so as soon as we have an issue that scratches below the surface, it turns into an all-hands-on-deck mayday situation, looping in other teams/experts to try and save them. I've also seen horrific implementations by abstraction enjoyers. Imagine a continuous integration environment of the following - k8s where one pod simply runs a forever shell script that is basically "git checkout master;git pull; sleep 300;". Then the other pod actually runs the application on top of the shared storage that the first pod writes to. This script, unsurprisingly, fails silently and hangs frequently. To debug this you need to go through some cloud auth portal and click through some web UI to then open a console in your browser (which doesn't support copy/paste). The idiot version of doing this on-prem would be a single small VM that has a cron running every 5 minutes, which invokes your deployment script, with cron configured to email on failure. This would be simpler, more reliable (not relying on a forever script), and more supportable (it emails on fail!). And it uses like 30 year old tech that just fkn works.

detourdog 3y ago

I totally relate to your point of view and experience. I'm so happy to ignore all the software related to the attention economy. What I find left over is the computing environment that I dreamt of when I was 12. I marvel that it took a whole generation of man hours to get to this point. I think we are real inflection point between an analog world and a digital world. I also think that the Bronze Age never went away and analog techniques will just develop into craftsmanship. I haven't posted a link to these Autonetics parts in while so I will do so again. What I love about Autonetics is that they were solving the inertial guidance problem in whatever computing form they could get a hold of. I'm working on a proper website but the images linked here represent the inertial guidance problem being solved with discrete components arranged in circuits and later those circuits integrated in silicon. https://en.wikipedia.org/wiki/Autonetics https://en.wikipedia.org/wiki/Autonetics https://www.icloud.com/sharedalbum/#B0YG4TcsmGWIVSf https://www.icloud.com/sharedalbum/#B0YG4TcsmGWIVSf Some Nixdorf 820 photos https://www.computerhistory.org/revolution/memory-storage/8/264/1116 https://www.computerhistory.org/revolution/memory-storage/8/... https://www.icloud.com/sharedalbum/#B0Y59UlCqNHCjx https://www.icloud.com/sharedalbum/#B0Y59UlCqNHCjx

yterdy 3y ago

Older workers are supposed to pass along this knowledge to younger workers in an expedited fashion. Acquisition of the understanding of increasingly complex systems must be optimized to operational needs. One of the many, many gripes Millennials have with Baby Boomers and Gen-Xers is that they jealously guard institutional knowledge in the name of job security, for years, and then just... retire. If you're lucky, your company will shell out massive consultation fees to have them come back and drip-feed what they know to younger personnel. It's not just the natural order of things that such knowledge passes out of this world. It's one of many collective and affirmative decisions by people born before 1980 to take as much with them as possible when they go. I would be skeptical if it were just one thing, but after affordable college, affordable housing, affordable healthcare, affordable retirement, and a viable biosphere, it's become a pattern. That you seem to understand this on some level and don't care to try to mke a difference is even worse.

lolinder 3y ago

An important thing to remember is that not only did you get to watch the layers be added, you also have decades of experience. In today's field, where we double in number every N years, that's enough time to accumulate enough knowledge to feel like a demigod—to have more than 20 years of experience is extremely rare in the industry right now. That won't always be the case: eventually we'll stop growing exponentially and we'll find an equilibrium, and eventually the people with 30 years of experience will stop being a vanishingly small minority. While that may not give us the kind of breadth that your generation has, I would expect many of them to be able to master the specific slice of the tech tree that they actually build on top of. 30 years is plenty of time to learn the everything from the browser down to the CPU, or the HTTP framework down, or the database down, and probably enough to learn two such slices. Part of why it seems impossible now is just because most developers not only don't have the historical context, they haven't yet had any time.

convolvatron 3y ago
its more than just time. the sad part about this evoluation is that younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago. i think whats required to move forward isn't just some dusty studying of these physics, but a reawakening to an understanding that these are just systems with tradeoffs, and we can choose to make other systems just as easily. thats the only way the sandpile collapses and we get .. for example .. secure operating systems and network protocols. or systems that were designed to exploit the massive amount of concurrency available in modern machines.
- lolinder 3y ago
  > younger people are enculturated to believe that the foundations that they are standing on represent some sort of inviolate physics and not just some technical decision someone made 20 years ago. This is simply not my experience of young people or of being young. There's a reason why radical movements are led and populated largely by the young—to the extent they err, it's on the side of "let's tear it all down and start over".

NohatCoder 3y ago

For the last 10 years you have been adding stuff that did nothing but increase the complexity. Now you wonder why new people can't keep up with the complexity.

tight-ship 3y ago

> perseverance to really understand all those things by diving into obscure, historical manuals. You say you were lucky to be there in the beginning and that young people don't have this advantage. Then you conflate your own luck with perseverance.

onetimeuse92304 3y ago
I am not conflating anything. For me it was my day to day work for decades. Some new technology happened and I had to learn it to be on top of my game. Trying to learn all this in 1/10th of the time to start your career is what requires perseverance.
- tight-ship 3y ago
  May I quote? > We, *people who really understand the technology* that the world runs on, are a slowly dying breed Thus one has to conclude that young people, not having perseverance, do not really understand technology (and lacking such basic skill, never will).

bbarn 3y ago

I think you may be confusing what you see with the cause. For example: You and I HAD to be willing to go through that, and we were fascinated by it. There are still plenty of great FE and BE young devs who see something and instantly ask "How does this work" and keep going down. The difference is, most people in employment don't HAVE to do this like all people in our early career did. If those shortcuts to getting paid existed in our early careers, many of us would have taken the money and not cared. It's not an experience or a "You had to be there" thing, it's simple curiosity and motivation that sets people apart in this regard.

sgarland 3y ago
Precisely this. Some people have an innate intellectual curiosity. Some do not. I do not enjoy working with the latter.

konschubert 3y ago

Which is fine, it's specialisation. Of course, it's bad if the western world fully outsources certain components of the stack. But not everybody needs to understand the full stack. Maybe you don't know how the silicon crystals are made and that is okay.

nonrandomstring 3y ago

> We, people who really understand the technology that the world runs on, are a slowly dying breed. We are still here as tech leads, managers, directors, business owners. But there will be a point in time when we will go on retirement and there will be only precious few people who had perseverance to really understand all those things by diving into obscure, historical manuals Some people like to say this is a perennial problem that every generation bemoans. I get why that might seem intuitive, and for the longest time I've hoped they're right and I'm merely getting old. Sadly they are wrong. Long term historical patterns of empire collapse follow this over-reach and under-education cycle that leads to a catastrophic rebuild capability gap that is triggered at some point. Thomas Thwaite's "The toaster project" [0] is a wonderful commentary on this. I cited it in a talk I did about education in world that will no longer pay for teaching kids how to use chips and breadboards in skinflint universities that prefer to teach them Microsoft Excel or use some crappy simulators of everything [1]. When it comes to cybersecurity we can only obtain defence in depth if we have knowledge in depth, and there are ever fewer of us around who understand how computers actually work. It's more than a little unnerving to meet CISOs and folks with high-flying titles who have no idea why using a phone app to control a Fortune 500 company infra might not be a good idea. [0] http://www.thetoasterproject.org/ http://www.thetoasterproject.org/ [1] https://www.youtube.com/watch?v=AxoSskalOrM https://www.youtube.com/watch?v=AxoSskalOrM

lr4444lr 3y ago

I suspect this happens in all technical professions as things advance: you learn first principles and low level implementations in a formal training program/degree, but the demands of the industry to be productive require that you use and trust abstractions. Even my doctors don't know some things about various drugs they prescribe me, because I can afford to spend the time reading the literature that only pertains to me. I'm sure their biochemistry background means they could definitely outperform me if they wanted to do a deep dive, but they know the important parts, and they know how my condition stacks up in the compendium of total knowledge and clinical experience they have. I don't think anything important is lost if they cannot at the drop of a hat tell me the half life and mechanism of action if I grill them on it.

dimmke 3y ago

This is the perfect articulation of something I think about all the time. It is so much easier if you were in it while it was happening. I've been trying to write a blog post about this, and how to address the problem specifically for front end development, where it's absolutely ludicrous. It's a very serious problem, especially when you think about security. You have all these people clamoring to get cloud certifications so they can work in IT but its so complex. A certification isn't enough. There just aren't enough people with enough knowledge and practical experience to fill all the needs that companies have.

deleted 3y ago

[deleted]

SubjectToChange 3y ago

I understand pretty much all of the layers of the computing from how CPUs achieve some of the things they are doing to bus protocols, to instructions, physical memory, low level OS internals, high level OS internals, virtual memory, userspace platform communication with OS, programming language runtimes and linking, shared libraries, IPC, networking, virtualization, etc. If by "understand" you mean "have a passing familiarity with" then.. sure, but if you mean "my understanding is reflective of reality" then there's absolutely no way. We, people who really understand the technology that the world runs on, are a slowly dying breed. There are more people doing "low level" programming right now than were doing it a few decades ago. The only knowledge that is fading is knowledge that's applicable to legacy and/or retro systems.