10 ms·

The fact that a library discourages you from learning it's internals is how you know it's well done. If you needed to understand it, you could. The reason you

The fact that a library discourages you from learning it's internals is how you know it's well done. If you needed to understand it, you could. The reason you don't is because you've never had a compelling reason to. If only one person knows how a library works, that is a problem. If a 100 person team maintains it and you're not on that team.... it's probably because you have other stuff to do. Software is an engineering discipline. Computer science is a science. If you are in programming because you want to advance our understanding, great, go work in one of the many fields with large novel algorithms that need to be understood. For a typical programmer... Look around. Modern software is one of the best things about the modern world. It does SO much for us. Do you really think we, with a real world distribution of programmer abilities, could do all this without massive division of labor? How much would be missing if we insisted on understanding everything before we use it? I suspect very few alternative approaches to software would work as well to truly build the modern software defined world where everything from cars to taxes to photography is digital. Because... whenever someone tries an alternative approach, there usually seems to be the hidden unspoken assumptions that they actually don't want software to be as big as it is. The end product of software itself these days is a service(in the sense of this article, not the SASS sense), wheras software built with an understanding-first mindset seems to usually wind up being closer to a digital version of a passive analog tool.

imbnwa 5y ago
"Division of labor is an extremely mature state for a society. Aiming prematurely for it is counterproductive. Rather than try to imitate more mature domains, start from scratch and see what this domain ends up needing." This is more the gist of the article than "lets not have libraries or abstractions at all", its pointing more towards "lets defer those questions until the right time rather than from the jump". I think Enterprise Java and the insane complexity of modern web dev are indicative of a consequence. >Because... whenever someone tries an alternative approach, there usually seems to be the hidden unspoken assumptions that they actually don't want software to be as big as it is. I really don't think that OP is making this assumption
- hinkley 5y ago
  People’s opinions on this stuff contain a lot of unstated pain points. Monolith first argues against a coping mechanism people use because they hate Big Ball of Mud projects. Anyone who has ever done repair work can tell you how often people wait until it’s far too late to do basic maintenance. Software is similarly bimodal. People either run it till it falls apart, or they baby it like a collector’s item. We have not collectively learned the trick of stopping at the first signs of pain or “strange noises” and figuring out what’s going on. But mostly we have not learned the trick of insisting to the business that we are doing our job when we do this, instead of plowing on blindly for six to eighteen months until there’s a break in the schedule. By which time everything’s a mess and two of the people you hoped would help you clean it up have already quit.
  - Frost1x 5y ago
    I think most software engineers know when it's time for maintenance. The issue is that the people who write their paychecks don't want to hear it.
    - hinkley 5y ago
      I read too much commit history, including my own, to agree with that. We are too close to the problem, and we enjoy deep dives too much. I save myself a lot of frustrating work by stopping for a cup of coffee or while brushing my teeth or washing my hair. “If I change Z I don’t need to touch X or Y”
      - Frost1x 5y ago
        I mean, I do the exact same (I did it Friday), but I don't think this is counter to my point. It's not because I don't want to fix something so I avoid it, it's because fixing it puts me behind on other tasks (that come down from people writing my paycheck), so I avoid tasks that won't be clearly rewarded unless I can convince them otherwise. It's because time pressures are designed in a way that incentivizes you not to do maintenance when you see a mess. It's also not sexy on a resume that you altered something from a mess to not a mess vs implementing some new feature. It takes highly detail oriented and creative people to develop good software and those traits tend to drive one crazy to the point of fixing something when you see a mess. Given no constraints I bet most developers would clean up their code to the best of their ability and fix things as they come across issues they identify. I've been in these no constraint environments, usually on stuff I write myself for myself, and I don't mind going back and doing a significant refactor when it's clearly needed. Once I'm done, I feel genuinely satisfied that I've done something useful and productive because I only need to convince myself and because no real external financial pressures exist in this context.
akkartik 5y ago
> The fact that a library discourages you from learning it's internals is how you know it's well done. If you needed to understand it, you could. The reason you don't is because you've never had a compelling reason to. I don't think we need to protect people from learning internals, they do that just fine on their own. I know many situations where we failed to understand internals even in the presence of compelling reason. With apologies for continuing to quote myself: "I want to carve steps into the wall of the learning process. Most programs today yield insight only after days or weeks of unrewarded effort. I want an hour of reward for an hour (or three) of effort." -- http://akkartik.name/about http://akkartik.name/about
- marktani 5y ago
  what is an hour of reward? I understand reward in terms of some form of a joyful/insightful/... experience, not in terms of time
  - akkartik 5y ago
    I think an older version said, "an hour's worth of reward." Does that seem clearer? Thanks.
  - coldtea 5y ago
    Parent talks about how fast you get back results from effort (learning, fiddling, etc) spent: "Most programs today yield insight only after days or weeks of unrewarded effort. I want an hour of reward for an hour (or three) of effort." So they want to be rewarded with at least an hour of saved time, or more productive time, for one (or few) hours they put in, instead of spending weeks to reap rewards.
    - akkartik 5y ago
      I was just alluding to a subjective sense that you got something reasonable for your trouble and efforts. You could do something, or you learned something, or you were entertained. My argument doesn't depend too much on the precise meaning, I think.
omegalulw 5y ago
> Do you really think we, with a real world distribution of programmer abilities, could do all this without massive division of labor? How much would be missing if we insisted on understanding everything before we use it? I agree that you don't want to spend time becoming super familiar with everything that you use but you should ALWAYS have a high level idea of what's happening under the hood. When you don't it inevitable leads to garbage in garbage out.
- fuzzfactor 5y ago
  >real world distribution of programmer abilities Way before the arrival of personal computers, it was clear for me to realize that I would be highly disappointed if average programmer abilities declined half as far as they have by now. Some of the most coherent technical projects are the result of a single individual's outstanding vision. Occasionally, the project can be brought to completion by that one individual, in ways that simply could not be exceeded alternatively, all the way from from fundamental algorithms through coding to UX and design. Additional engineers could be considered useful or essential simply to accelerate the completion or launch. When the second engineer is brought on board, it may be equally performant to have them come fully up to speed on everything the first engineer is doing, compared to having them concentrate on disparate but complementary needs which need to be brought to the table in addition. If they both remain fully versed, then either one can do anything that comes up. Sometimes they can work on the same thing together when needed, other times doing completely different things to make progress in two areas simultaneously. You're going to end up with the same product either way, the second engineer just allows an earlier launch date. But truly never within half the calendar time, those man-months are legendarily mythical. For projects beyond a certain size more engineers are simply essential. Then ask yourself what kind of team would you rather have? All engineers who could contribute to any part of the whole enchilada from primordial logic through unexceeded UI/UX? Or at the opposite end of the spectrum, all engineers who are so specialized that the majority of them have no involvement whatsoever with things like UI/UX? Assuming you're delivering as satisfactory a product as can be done by the same number of engineers in the same time frame. You're NOT going to end up with the same product either way. Now aren't you glad it's a spectrum so it gives you an infinitely greater number of choices other than the optimum?
nitwit005 5y ago
Few library and service creators seem to think you can treat what they built as a black box. At the very least, they generally come with some advice on usage patterns to avoid. Writing performant code unfortunately tends to require at least having a basic working model of how all your dependencies work. Otherwise, you tend to find yourself debugging why you ran out of file handles in production.
- edgyquant 5y ago
  Well I think that’s because most open sources licenses remove any liability from the creators and have no support outside of volunteers. For this reason a lot of them, and even on closed source projects to add an extra layer before support, provide basic troubleshooting information.
- kristov 5y ago
  Most (useful) libraries are managing state. This means the user of the library should have a pretty good idea of how the state is managed or they will call functions in the wrong order.
deleted 5y ago
[deleted]
edgyquant 5y ago
Programming seems to be one of the few fields where we think it’s a bad thing if our tools aren’t made by us (this has luckily been waning in recent years, but the push back against “bloat” we see on every tech forum is proof that the mindset is entrenched.
andresp 5y ago
Exactly. Division of labor is essential for any complex project. Whoever says the opposite has probably only worked in simple/small systems.
eternityforest 5y ago
UPDATE: It seems the good folks at a certain subreddit found this comment and are interpreting "Discourage you from learning about it" as "Make it copyrighted" or obfusctation or something. I don't think those are signs of a good library! I'm referring more to things like FOSS JS frameworks with the kind of batteries included design that prioritizes abstraction and encapsulation over simplicity. Nothing actually stops you from learning them, it just takes time, because their big, and it's not necessary to use them.
newshorts 5y ago
I have to agree. The article quotes sound right and wonderful, but thinking about the environment in which I work, there’s not really a practical way to combine our efforts. We simply wouldn’t get things done in time. Division of labor ensures that changes can happen reliably and quickly. I’m ramping on an internal library right now and it’s taking a while to understand it’s complexities and history. I can’t imagine making quick changes to this without breaking a lot of stuff. It will take time to develop an intuition for predicting a change’s impact. Now multiply my ramp up time by every dev on my team every time they need to make an update. You can imagine productivity slows down and cognitive overhead rises.
analog31 5y ago
I actually think this applies to the passive analog world too. Very few engineers could tell you how steel or concrete are made. And we all use abstractions / libraries, for instance using tabulated properties of steel alloys rather than knowing the underlying physics. In fact, if put in plainer terms, every engineer would nod in agreement with Hyrum's Law. Everybody has a story of a design that worked because of an undocumented property of a material, that stopped working when Purchasing changed to a new supplier of that material.
- xorcist 5y ago
  On the contrary. Every engineer is able to learn the process when it matters. And it often does. The type of steel used, the treatment and the orientation is imporant to the outcome. You can not hand wave them away as "implementation details".
  - davidgay 5y ago
    > Every engineer is able to learn the process when it matters. A civil engineer is going to become an expert material scientist, "on need"? I doubt it.
    - xorcist 5y ago
      Of course not. A basic understanding? Absolutely, yes. It's even widely accepted to be an important part of the education. Just as Karnaugh maps and parser theory is part of computer science engineering curriculums. It's not something that's expected to be used a daily basis but some general knowledge of how a processor works is absolutely necessary to be able to at least talk about cache line misses, performance counters, mode swaps etc.
      - analog31 5y ago
        One issue is that the divisions between engineering fields are somewhat arbitrary, and technology doesn't always respect those divisions, so we don't know in advance what education is going to be needed. A second problem is that we make it very hard for young engineers to maintain their technical chops when they can be quite busy and productive doing basic design. In fact, engineering students hear through the grapevine: "You won't use this stuff once you get a job." As a result, the industry settles on an "efficient" way of managing issues that require depth or breadth, which has to have a few people around who handle those things when needed. That becomes a form of division-of-labor.

imbnwa 5y ago

"Division of labor is an extremely mature state for a society. Aiming prematurely for it is counterproductive. Rather than try to imitate more mature domains, start from scratch and see what this domain ends up needing." This is more the gist of the article than "lets not have libraries or abstractions at all", its pointing more towards "lets defer those questions until the right time rather than from the jump". I think Enterprise Java and the insane complexity of modern web dev are indicative of a consequence. >Because... whenever someone tries an alternative approach, there usually seems to be the hidden unspoken assumptions that they actually don't want software to be as big as it is. I really don't think that OP is making this assumption

hinkley 5y ago
People’s opinions on this stuff contain a lot of unstated pain points. Monolith first argues against a coping mechanism people use because they hate Big Ball of Mud projects. Anyone who has ever done repair work can tell you how often people wait until it’s far too late to do basic maintenance. Software is similarly bimodal. People either run it till it falls apart, or they baby it like a collector’s item. We have not collectively learned the trick of stopping at the first signs of pain or “strange noises” and figuring out what’s going on. But mostly we have not learned the trick of insisting to the business that we are doing our job when we do this, instead of plowing on blindly for six to eighteen months until there’s a break in the schedule. By which time everything’s a mess and two of the people you hoped would help you clean it up have already quit.
- Frost1x 5y ago
  I think most software engineers know when it's time for maintenance. The issue is that the people who write their paychecks don't want to hear it.
  - hinkley 5y ago
    I read too much commit history, including my own, to agree with that. We are too close to the problem, and we enjoy deep dives too much. I save myself a lot of frustrating work by stopping for a cup of coffee or while brushing my teeth or washing my hair. “If I change Z I don’t need to touch X or Y”
    - Frost1x 5y ago
      I mean, I do the exact same (I did it Friday), but I don't think this is counter to my point. It's not because I don't want to fix something so I avoid it, it's because fixing it puts me behind on other tasks (that come down from people writing my paycheck), so I avoid tasks that won't be clearly rewarded unless I can convince them otherwise. It's because time pressures are designed in a way that incentivizes you not to do maintenance when you see a mess. It's also not sexy on a resume that you altered something from a mess to not a mess vs implementing some new feature. It takes highly detail oriented and creative people to develop good software and those traits tend to drive one crazy to the point of fixing something when you see a mess. Given no constraints I bet most developers would clean up their code to the best of their ability and fix things as they come across issues they identify. I've been in these no constraint environments, usually on stuff I write myself for myself, and I don't mind going back and doing a significant refactor when it's clearly needed. Once I'm done, I feel genuinely satisfied that I've done something useful and productive because I only need to convince myself and because no real external financial pressures exist in this context.

akkartik 5y ago

> The fact that a library discourages you from learning it's internals is how you know it's well done. If you needed to understand it, you could. The reason you don't is because you've never had a compelling reason to. I don't think we need to protect people from learning internals, they do that just fine on their own. I know many situations where we failed to understand internals even in the presence of compelling reason. With apologies for continuing to quote myself: "I want to carve steps into the wall of the learning process. Most programs today yield insight only after days or weeks of unrewarded effort. I want an hour of reward for an hour (or three) of effort." -- http://akkartik.name/about http://akkartik.name/about

marktani 5y ago
what is an hour of reward? I understand reward in terms of some form of a joyful/insightful/... experience, not in terms of time
- akkartik 5y ago
  I think an older version said, "an hour's worth of reward." Does that seem clearer? Thanks.
- coldtea 5y ago
  Parent talks about how fast you get back results from effort (learning, fiddling, etc) spent: "Most programs today yield insight only after days or weeks of unrewarded effort. I want an hour of reward for an hour (or three) of effort." So they want to be rewarded with at least an hour of saved time, or more productive time, for one (or few) hours they put in, instead of spending weeks to reap rewards.
  - akkartik 5y ago
    I was just alluding to a subjective sense that you got something reasonable for your trouble and efforts. You could do something, or you learned something, or you were entertained. My argument doesn't depend too much on the precise meaning, I think.

omegalulw 5y ago

> Do you really think we, with a real world distribution of programmer abilities, could do all this without massive division of labor? How much would be missing if we insisted on understanding everything before we use it? I agree that you don't want to spend time becoming super familiar with everything that you use but you should ALWAYS have a high level idea of what's happening under the hood. When you don't it inevitable leads to garbage in garbage out.

fuzzfactor 5y ago
>real world distribution of programmer abilities Way before the arrival of personal computers, it was clear for me to realize that I would be highly disappointed if average programmer abilities declined half as far as they have by now. Some of the most coherent technical projects are the result of a single individual's outstanding vision. Occasionally, the project can be brought to completion by that one individual, in ways that simply could not be exceeded alternatively, all the way from from fundamental algorithms through coding to UX and design. Additional engineers could be considered useful or essential simply to accelerate the completion or launch. When the second engineer is brought on board, it may be equally performant to have them come fully up to speed on everything the first engineer is doing, compared to having them concentrate on disparate but complementary needs which need to be brought to the table in addition. If they both remain fully versed, then either one can do anything that comes up. Sometimes they can work on the same thing together when needed, other times doing completely different things to make progress in two areas simultaneously. You're going to end up with the same product either way, the second engineer just allows an earlier launch date. But truly never within half the calendar time, those man-months are legendarily mythical. For projects beyond a certain size more engineers are simply essential. Then ask yourself what kind of team would you rather have? All engineers who could contribute to any part of the whole enchilada from primordial logic through unexceeded UI/UX? Or at the opposite end of the spectrum, all engineers who are so specialized that the majority of them have no involvement whatsoever with things like UI/UX? Assuming you're delivering as satisfactory a product as can be done by the same number of engineers in the same time frame. You're NOT going to end up with the same product either way. Now aren't you glad it's a spectrum so it gives you an infinitely greater number of choices other than the optimum?

nitwit005 5y ago

Few library and service creators seem to think you can treat what they built as a black box. At the very least, they generally come with some advice on usage patterns to avoid. Writing performant code unfortunately tends to require at least having a basic working model of how all your dependencies work. Otherwise, you tend to find yourself debugging why you ran out of file handles in production.

edgyquant 5y ago
Well I think that’s because most open sources licenses remove any liability from the creators and have no support outside of volunteers. For this reason a lot of them, and even on closed source projects to add an extra layer before support, provide basic troubleshooting information.
kristov 5y ago
Most (useful) libraries are managing state. This means the user of the library should have a pretty good idea of how the state is managed or they will call functions in the wrong order.

deleted 5y ago

[deleted]

edgyquant 5y ago

Programming seems to be one of the few fields where we think it’s a bad thing if our tools aren’t made by us (this has luckily been waning in recent years, but the push back against “bloat” we see on every tech forum is proof that the mindset is entrenched.

andresp 5y ago

Exactly. Division of labor is essential for any complex project. Whoever says the opposite has probably only worked in simple/small systems.

eternityforest 5y ago

UPDATE: It seems the good folks at a certain subreddit found this comment and are interpreting "Discourage you from learning about it" as "Make it copyrighted" or obfusctation or something. I don't think those are signs of a good library! I'm referring more to things like FOSS JS frameworks with the kind of batteries included design that prioritizes abstraction and encapsulation over simplicity. Nothing actually stops you from learning them, it just takes time, because their big, and it's not necessary to use them.

newshorts 5y ago

I have to agree. The article quotes sound right and wonderful, but thinking about the environment in which I work, there’s not really a practical way to combine our efforts. We simply wouldn’t get things done in time. Division of labor ensures that changes can happen reliably and quickly. I’m ramping on an internal library right now and it’s taking a while to understand it’s complexities and history. I can’t imagine making quick changes to this without breaking a lot of stuff. It will take time to develop an intuition for predicting a change’s impact. Now multiply my ramp up time by every dev on my team every time they need to make an update. You can imagine productivity slows down and cognitive overhead rises.

analog31 5y ago

I actually think this applies to the passive analog world too. Very few engineers could tell you how steel or concrete are made. And we all use abstractions / libraries, for instance using tabulated properties of steel alloys rather than knowing the underlying physics. In fact, if put in plainer terms, every engineer would nod in agreement with Hyrum's Law. Everybody has a story of a design that worked because of an undocumented property of a material, that stopped working when Purchasing changed to a new supplier of that material.

xorcist 5y ago
On the contrary. Every engineer is able to learn the process when it matters. And it often does. The type of steel used, the treatment and the orientation is imporant to the outcome. You can not hand wave them away as "implementation details".
- davidgay 5y ago
  > Every engineer is able to learn the process when it matters. A civil engineer is going to become an expert material scientist, "on need"? I doubt it.
  - xorcist 5y ago
    Of course not. A basic understanding? Absolutely, yes. It's even widely accepted to be an important part of the education. Just as Karnaugh maps and parser theory is part of computer science engineering curriculums. It's not something that's expected to be used a daily basis but some general knowledge of how a processor works is absolutely necessary to be able to at least talk about cache line misses, performance counters, mode swaps etc.
    - analog31 5y ago
      One issue is that the divisions between engineering fields are somewhat arbitrary, and technology doesn't always respect those divisions, so we don't know in advance what education is going to be needed. A second problem is that we make it very hard for young engineers to maintain their technical chops when they can be quite busy and productive doing basic design. In fact, engineering students hear through the grapevine: "You won't use this stuff once you get a job." As a result, the industry settles on an "efficient" way of managing issues that require depth or breadth, which has to have a few people around who handle those things when needed. That becomes a form of division-of-labor.