6 ms·

This is a false dichotomy. The truth is we are constantly moving further and further away from the silicon. New developers don't have as much need to understand

This is a false dichotomy. The truth is we are constantly moving further and further away from the silicon. New developers don't have as much need to understand these details because things just work; some do care because they work at a job where it's required, or because they're inherently interested (a small number). Over time we will move further away. If the cost of an easily managed solution is low enough, why do the details matter?

lifeisstillgood 2y ago
I used to think this, ut it only works if the abstractions hold - it’s like if we stopped random access memory and went back to tape drives suddenly abstractions matter. My comment elsewhere goes into but more detail but basically silicon stopped being able to make single threaded code faster in about 2012 - we just have been getting “more parallel cores” since. And now at wafer scale we see 900,000 cores on a “chip”. When 100% parallel coding runs 1 million times faster than your competitors, when following one software engineering path leads to code that can run 1M X, then we will find ways to use that excess capacity - and the engineers who can do it get to win. I’m not sure how LLMs face this problem.
- jonas21 2y ago
  That's why abstractions like PyTorch exist. You can write a few lines of Python and get good utilization of all those GPU cores.
  - varelse 2y ago
    [dead]
- dbcjv7vhxj 2y ago
  This. As soon as the abstractions leak or you run into an underlying issue you suddenly need to understand everything about the underlying system or you're SOOL. I'd rather have a simpler system I already understand all the proceeding abstractions about. The overhead of this is minimal when you keep things simple and avoid shiny things.
serviceberry 2y ago
> The truth is we are constantly moving further and further away from the silicon. Are we? We're constantly changing abstractions, but we don't keep adding them all that often. Operating systems and high-level programming languages emerged in the 1960s. Since then, the only fundamentally new layer of abstraction were virtual machines (JVM, browser JS, hardware virtualization, etc). There's still plenty of hardware-specific APIs, you still debug assembly when something crashes, you still optimize databases for specific storage technologies and multimedia transcoders for specific CPU architectures...
- fphhotchips 2y ago
  > There's still plenty of hardware-specific APIs, you still debug assembly when something crashes, you still optimize databases for specific storage technologies and multimedia transcoders for specific CPU architectures... You might, maybe, but an increasing proportion of developers: - Don't have access to the assembly to debug it - Don't even know what storage tech their database is sitting on - Don't know or even control what CPU architecture their code is running on. My job is debugging and performance profiling other people's code, but the vast majority of that is looking at query plans. If I'm really stumped, I'll look at the C++, but I've not yet once looked at assembly for it.
  - daveguy 2y ago
    This makes sense to me. When I optimize, the most significant gains I find are algorithmic. Whether it's an extra call, a data structure that needs to be tweaked, or just utilizing a library that operates closer to silicon. I rarely need to go to assembly or even a lower level language to get acceptable performance. The only exception is occasionally getting into architecture specifics of a GPU. At this point, optimizing compilers are excellent and probably have more architecture details baked into them than I will ever know. Thank you, compiler programmers!
    - davemp 2y ago
      > At this point, optimizing compilers are excellent and probably have more architecture details baked into them than I will ever know. While modern compilers are great, you’d be surprised about the seemingly obvious optimizations compilers can’t do because of language semantics or the code transformations would be infeasible to detect. I type versions of functions into godbolt all the time and it’s very interesting to see what code is/isn’t equivalent after O3 passes
layoric 2y ago
Capture and product stickiness. If your product is all serverless wired together with an event system by the same cloud provider, you are in a very weak position to argue that you will go elsewhere where, leveraging the competitive market to your advantage. The more the big cloud providers can abstract cpu cycles, memory, networking, storage etc, the more they don’t have to compete with others doing the same.
cathalc 2y ago
"Preventing the Collapse of Civilization" by Jonathan Blow comes to mind - https://www.youtube.com/watch?v=ZSRHeXYDLko https://www.youtube.com/watch?v=ZSRHeXYDLko A fantastic talk.
Uvix 2y ago
The details matter because someone has to understand the details, and it's quicker and more cost-effective if it's the developer. At my job, a decade ago our developers understood how things worked, what was running on each server, where to look if there were problems, etc. Now the developers just put magic incantations given to them by the "DevOps team" into their config files. Most of them don't understand where the code is running, or even what much of it is doing. They're unable or unwilling to investigate problems on their own, even if they were the cause of the issue. Even getting them to find the error message in the logs can be like pulling teeth. They rely on this support team to do the investigation for them, but continually swiveling back-and-forth is never going to be as efficient as when the developer could do it all themselves. Not to mention it requires maintaining said support team, all those additional salaries, etc. (I'm part of said support team, but I really wish we didn't exist. We started to take over Ops responsibilities from a different team, but we ended up taking on Dev ones too and we never should've done that.)
- busterarm 2y ago
  > (I'm part of said support team, but I really wish we didn't exist. We started to take over Ops responsibilities from a different team, but we ended up taking on Dev ones too and we never should've done that.) There will always be work for people like us. It's not so bad. We're not totally immune to layoffs but for us they come several rounds in.
- mandevil 2y ago
  http://www.antipope.org/charlie/blog-static/2014/10/not-a-manifesto.html http://www.antipope.org/charlie/blog-static/2014/10/not-a-ma... This blog has a brilliant insight that I still remember more than a decade later: we live in a fantasy setting, not a Sci-fi one. Our modern computers are so unfathomable complex that they are demons, ancient magic that can be tamed and barely manipulated, but not engineered. Modern computing isn't Star Trek TNG, where Captain Picard and Geordi LaForge each have every layer of their starship in their heads with full understanding, and they can manipulate each layer independently. We live in a world where the simple cell phone in our pocket contains so much complexity that it is beyond any 10 human minds combined to fully understand how the hardware, the device drivers, the OS, the app layer, and the internet all interact between each other.
swatcoder 2y ago
> because things just work If that were true, you might be right. What happens in reality is that things are promised to work and (at best) fulfill that promise so long as no developers or deployers or underlying systems or users deviate from a narrow golden path, but fail in befuddling ways when any of those constraints introduce a deviation. And so what we see, year over year, is continued enshittening, with everything continuously pushing the boundaries of unreliability and inefficiency, and fewer and fewer people qualified to actually dig into the details to understand how these systems work, how to diagnose their issues, how to repair them, or how to explain their costs. > If the cost of an easily managed solution is low enough, why do the details matter? Because the patience that users have for degraded quality, and the luxury that budgets have for inefficiency, will eventually be exhausted and we'll have collectively led ourselves into a dark forest nobody has the tools or knowledge to navigate out of anymore. Leveraging abstractions and assembling things from components are good things that enable rapid exploration and growth, but they come with latent costs that eventually need to be revisited. If enough attention isn't paid too understanding, maintaining, refining, and innovating on the lowest levels, the contraptions built through high-level abstraction and assempbly will eventually either collapse upon themselves or be flanked by competitors who struck a better balance and built on more refined and informed foundations. As a software engineer who wants a long and satisfying career, you should be seeking to understand your systems to as much depth as you can, making informed, contextual choices about what abstractions you leverage, exactly what they abstract over, and what vulnerabilities and limitations are absorbed into your projects by using them. Just making naive use of the things you found a tutorial for, or that are trending, or that make things look easy today, is a poison to your career.
pdntspa 2y ago
> why do the details matter? This statement encapsulates nearly everything that I think is wrong with software development today. Captured by MBA types trying to make a workforce that is as cheap and replaceable as possible. Details are simply friction in a machine that is obsessed with efficiency to the point of self-immolation. And yet that is the direction we are moving in. Details matter, process matters, experience and veterancy matters. Now more than ever.
llm_trw 2y ago
> If the cost of an easily managed solution is low enough Because vertical scaling is now large enough that I can run all of twitter/amazon on one single large server. And if I'm wrong now, in a decade I won't be. Compute power grows exponentially, but business requirements do not.
Spooky23 2y ago
I think that’s the same answer someone would say about an IBM mainframe in 1990. And just as wrong. I’ll use my stupid hobby home server stuff as an example. I tossed the old VMware box years ago. You know what I use now? Little HP t6x0 thin clients. They are crappy little x86 SoCs with m2 slots, up to 32GB memory and they can be purchased used for $40. They aren’t fast, but perform better than the cheaper AWS and GCP instances. In that a trivial use case? Absolutely. Now move from $30 to about $2000. Buy a Mac Mini. It’s a powerful arm soc with ridiculously fast storage and performance. Probably more compute than a small/mid size company computer room a few years ago and more performant than a $1M SAN a decade ago. 6G will bring 10gig cellular. Hyperscalers datacenters are the mainframe of 2025.
- c0balt 2y ago
  A hyperscaler (or a cloud providers in general) does not only sell you compute in terms of a compute node but rather in compute as a service. There are some value adds, like e.g., AWS cloud services, but on a pure compute level you pay for elasticity and reliability. A comparison between a cloud provider and your homelab also needs to account for connectivity, which likely is in strong favor (latency/ reliability) of a cloud provider or DC compared to a office or home.
  - Spooky23 2y ago
    Absolutely — they add a ton of value. So did IBM… and companies migrated to NT solutions that were half baked because they were cheap. When I can get the equivalent of a Mac Mini in a super cheap price point… you’re going to have opportunities to attack those stratospheric cloud margins.
  - fpoling 2y ago
    Server hardware is reliable. For connectivity price-wise I think the sweet spot presently is to host own hardware in the data center and have a system administrator that lives not far away. I worked before for a company that was doing things like that while having millions of active users. It costed them at least 5 times less then it would be with a cloud provider. And then when they got a better deal with another data center the migration was not much more complex then moving server boxes in a van and changing ip addresses for load balancers.
sgarland 2y ago
> why do the details matter? Have you ever had a plumber, HVAC tech, electrician, etc. come out to your house for something, and had them explain it to you? Have you had the unfortunate experience of that happening more than once (with separate people)? If so, you should know why this matters: because if you don’t understand the fundamentals, you can’t possibly understand the entire system. It’s the same reason why the U.S. Navy Nuclear program still teaches Electronics Technicians incredibly low-level things like bus arbitration on a 386 (before that, it was the 68000). Not because they expect most to need to use that information (though if necessary, they carry everything down to logic analyzers), but because if you don’t understand the fundamentals, you cannot understand the abstractions. Actually, the CPU is an abstraction, I misspoke: they start by learning electron flow, then moving into PN junctions, then transistors, then digital logic, and then and only then do they finally learn how all of those can be put together to accomplish work. Incidentally, former Navy Nukes were on the initial Google SRE team. If you read the book [0], especially Chapter 12, you’ll get an inkling about why this depth of knowledge matters. Do most people need to understand how their NIC turns data into electrical signals? No, of course not. But occasionally, some weird bug emerges where that knowledge very much matters. At some point, most people will encounter a bug that they are incapable of reasoning about, because they do not possess the requisite knowledge to do so. When that happens, it should be a humbling experience, and ideally, you endeavor to learn more about the thing you are stuck on. [0]: https://sre.google/sre-book/table-of-contents/ https://sre.google/sre-book/table-of-contents/
zelon88 2y ago
Tell that to the people who keep the network gear running at your office. You might not see the importance of knowing the details, but those details still matter and are still in plain use all around you every day. The aversion to learning the stack you're building with is frustrating to the people who keep that stack running. I think that if the development side knew a little bit of the rest of the stack they'd write better applications overall.
- ChoHag 2y ago
  [dead]

lifeisstillgood 2y ago

I used to think this, ut it only works if the abstractions hold - it’s like if we stopped random access memory and went back to tape drives suddenly abstractions matter. My comment elsewhere goes into but more detail but basically silicon stopped being able to make single threaded code faster in about 2012 - we just have been getting “more parallel cores” since. And now at wafer scale we see 900,000 cores on a “chip”. When 100% parallel coding runs 1 million times faster than your competitors, when following one software engineering path leads to code that can run 1M X, then we will find ways to use that excess capacity - and the engineers who can do it get to win. I’m not sure how LLMs face this problem.

jonas21 2y ago
That's why abstractions like PyTorch exist. You can write a few lines of Python and get good utilization of all those GPU cores.
- varelse 2y ago
  [dead]
dbcjv7vhxj 2y ago
This. As soon as the abstractions leak or you run into an underlying issue you suddenly need to understand everything about the underlying system or you're SOOL. I'd rather have a simpler system I already understand all the proceeding abstractions about. The overhead of this is minimal when you keep things simple and avoid shiny things.

serviceberry 2y ago

> The truth is we are constantly moving further and further away from the silicon. Are we? We're constantly changing abstractions, but we don't keep adding them all that often. Operating systems and high-level programming languages emerged in the 1960s. Since then, the only fundamentally new layer of abstraction were virtual machines (JVM, browser JS, hardware virtualization, etc). There's still plenty of hardware-specific APIs, you still debug assembly when something crashes, you still optimize databases for specific storage technologies and multimedia transcoders for specific CPU architectures...

fphhotchips 2y ago
> There's still plenty of hardware-specific APIs, you still debug assembly when something crashes, you still optimize databases for specific storage technologies and multimedia transcoders for specific CPU architectures... You might, maybe, but an increasing proportion of developers: - Don't have access to the assembly to debug it - Don't even know what storage tech their database is sitting on - Don't know or even control what CPU architecture their code is running on. My job is debugging and performance profiling other people's code, but the vast majority of that is looking at query plans. If I'm really stumped, I'll look at the C++, but I've not yet once looked at assembly for it.
- daveguy 2y ago
  This makes sense to me. When I optimize, the most significant gains I find are algorithmic. Whether it's an extra call, a data structure that needs to be tweaked, or just utilizing a library that operates closer to silicon. I rarely need to go to assembly or even a lower level language to get acceptable performance. The only exception is occasionally getting into architecture specifics of a GPU. At this point, optimizing compilers are excellent and probably have more architecture details baked into them than I will ever know. Thank you, compiler programmers!
  - davemp 2y ago
    > At this point, optimizing compilers are excellent and probably have more architecture details baked into them than I will ever know. While modern compilers are great, you’d be surprised about the seemingly obvious optimizations compilers can’t do because of language semantics or the code transformations would be infeasible to detect. I type versions of functions into godbolt all the time and it’s very interesting to see what code is/isn’t equivalent after O3 passes

layoric 2y ago

Capture and product stickiness. If your product is all serverless wired together with an event system by the same cloud provider, you are in a very weak position to argue that you will go elsewhere where, leveraging the competitive market to your advantage. The more the big cloud providers can abstract cpu cycles, memory, networking, storage etc, the more they don’t have to compete with others doing the same.

cathalc 2y ago

"Preventing the Collapse of Civilization" by Jonathan Blow comes to mind - https://www.youtube.com/watch?v=ZSRHeXYDLko https://www.youtube.com/watch?v=ZSRHeXYDLko A fantastic talk.

Uvix 2y ago

The details matter because someone has to understand the details, and it's quicker and more cost-effective if it's the developer. At my job, a decade ago our developers understood how things worked, what was running on each server, where to look if there were problems, etc. Now the developers just put magic incantations given to them by the "DevOps team" into their config files. Most of them don't understand where the code is running, or even what much of it is doing. They're unable or unwilling to investigate problems on their own, even if they were the cause of the issue. Even getting them to find the error message in the logs can be like pulling teeth. They rely on this support team to do the investigation for them, but continually swiveling back-and-forth is never going to be as efficient as when the developer could do it all themselves. Not to mention it requires maintaining said support team, all those additional salaries, etc. (I'm part of said support team, but I really wish we didn't exist. We started to take over Ops responsibilities from a different team, but we ended up taking on Dev ones too and we never should've done that.)

busterarm 2y ago
> (I'm part of said support team, but I really wish we didn't exist. We started to take over Ops responsibilities from a different team, but we ended up taking on Dev ones too and we never should've done that.) There will always be work for people like us. It's not so bad. We're not totally immune to layoffs but for us they come several rounds in.
mandevil 2y ago
http://www.antipope.org/charlie/blog-static/2014/10/not-a-manifesto.html http://www.antipope.org/charlie/blog-static/2014/10/not-a-ma... This blog has a brilliant insight that I still remember more than a decade later: we live in a fantasy setting, not a Sci-fi one. Our modern computers are so unfathomable complex that they are demons, ancient magic that can be tamed and barely manipulated, but not engineered. Modern computing isn't Star Trek TNG, where Captain Picard and Geordi LaForge each have every layer of their starship in their heads with full understanding, and they can manipulate each layer independently. We live in a world where the simple cell phone in our pocket contains so much complexity that it is beyond any 10 human minds combined to fully understand how the hardware, the device drivers, the OS, the app layer, and the internet all interact between each other.

swatcoder 2y ago

> because things just work If that were true, you might be right. What happens in reality is that things are promised to work and (at best) fulfill that promise so long as no developers or deployers or underlying systems or users deviate from a narrow golden path, but fail in befuddling ways when any of those constraints introduce a deviation. And so what we see, year over year, is continued enshittening, with everything continuously pushing the boundaries of unreliability and inefficiency, and fewer and fewer people qualified to actually dig into the details to understand how these systems work, how to diagnose their issues, how to repair them, or how to explain their costs. > If the cost of an easily managed solution is low enough, why do the details matter? Because the patience that users have for degraded quality, and the luxury that budgets have for inefficiency, will eventually be exhausted and we'll have collectively led ourselves into a dark forest nobody has the tools or knowledge to navigate out of anymore. Leveraging abstractions and assembling things from components are good things that enable rapid exploration and growth, but they come with latent costs that eventually need to be revisited. If enough attention isn't paid too understanding, maintaining, refining, and innovating on the lowest levels, the contraptions built through high-level abstraction and assempbly will eventually either collapse upon themselves or be flanked by competitors who struck a better balance and built on more refined and informed foundations. As a software engineer who wants a long and satisfying career, you should be seeking to understand your systems to as much depth as you can, making informed, contextual choices about what abstractions you leverage, exactly what they abstract over, and what vulnerabilities and limitations are absorbed into your projects by using them. Just making naive use of the things you found a tutorial for, or that are trending, or that make things look easy today, is a poison to your career.

pdntspa 2y ago

> why do the details matter? This statement encapsulates nearly everything that I think is wrong with software development today. Captured by MBA types trying to make a workforce that is as cheap and replaceable as possible. Details are simply friction in a machine that is obsessed with efficiency to the point of self-immolation. And yet that is the direction we are moving in. Details matter, process matters, experience and veterancy matters. Now more than ever.

llm_trw 2y ago

> If the cost of an easily managed solution is low enough Because vertical scaling is now large enough that I can run all of twitter/amazon on one single large server. And if I'm wrong now, in a decade I won't be. Compute power grows exponentially, but business requirements do not.

Spooky23 2y ago

I think that’s the same answer someone would say about an IBM mainframe in 1990. And just as wrong. I’ll use my stupid hobby home server stuff as an example. I tossed the old VMware box years ago. You know what I use now? Little HP t6x0 thin clients. They are crappy little x86 SoCs with m2 slots, up to 32GB memory and they can be purchased used for $40. They aren’t fast, but perform better than the cheaper AWS and GCP instances. In that a trivial use case? Absolutely. Now move from $30 to about $2000. Buy a Mac Mini. It’s a powerful arm soc with ridiculously fast storage and performance. Probably more compute than a small/mid size company computer room a few years ago and more performant than a $1M SAN a decade ago. 6G will bring 10gig cellular. Hyperscalers datacenters are the mainframe of 2025.

c0balt 2y ago
A hyperscaler (or a cloud providers in general) does not only sell you compute in terms of a compute node but rather in compute as a service. There are some value adds, like e.g., AWS cloud services, but on a pure compute level you pay for elasticity and reliability. A comparison between a cloud provider and your homelab also needs to account for connectivity, which likely is in strong favor (latency/ reliability) of a cloud provider or DC compared to a office or home.
- Spooky23 2y ago
  Absolutely — they add a ton of value. So did IBM… and companies migrated to NT solutions that were half baked because they were cheap. When I can get the equivalent of a Mac Mini in a super cheap price point… you’re going to have opportunities to attack those stratospheric cloud margins.
- fpoling 2y ago
  Server hardware is reliable. For connectivity price-wise I think the sweet spot presently is to host own hardware in the data center and have a system administrator that lives not far away. I worked before for a company that was doing things like that while having millions of active users. It costed them at least 5 times less then it would be with a cloud provider. And then when they got a better deal with another data center the migration was not much more complex then moving server boxes in a van and changing ip addresses for load balancers.

sgarland 2y ago

> why do the details matter? Have you ever had a plumber, HVAC tech, electrician, etc. come out to your house for something, and had them explain it to you? Have you had the unfortunate experience of that happening more than once (with separate people)? If so, you should know why this matters: because if you don’t understand the fundamentals, you can’t possibly understand the entire system. It’s the same reason why the U.S. Navy Nuclear program still teaches Electronics Technicians incredibly low-level things like bus arbitration on a 386 (before that, it was the 68000). Not because they expect most to need to use that information (though if necessary, they carry everything down to logic analyzers), but because if you don’t understand the fundamentals, you cannot understand the abstractions. Actually, the CPU is an abstraction, I misspoke: they start by learning electron flow, then moving into PN junctions, then transistors, then digital logic, and then and only then do they finally learn how all of those can be put together to accomplish work. Incidentally, former Navy Nukes were on the initial Google SRE team. If you read the book [0], especially Chapter 12, you’ll get an inkling about why this depth of knowledge matters. Do most people need to understand how their NIC turns data into electrical signals? No, of course not. But occasionally, some weird bug emerges where that knowledge very much matters. At some point, most people will encounter a bug that they are incapable of reasoning about, because they do not possess the requisite knowledge to do so. When that happens, it should be a humbling experience, and ideally, you endeavor to learn more about the thing you are stuck on. [0]: https://sre.google/sre-book/table-of-contents/ https://sre.google/sre-book/table-of-contents/

zelon88 2y ago

Tell that to the people who keep the network gear running at your office. You might not see the importance of knowing the details, but those details still matter and are still in plain use all around you every day. The aversion to learning the stack you're building with is frustrating to the people who keep that stack running. I think that if the development side knew a little bit of the rest of the stack they'd write better applications overall.

ChoHag 2y ago
[dead]