11 ms·

One of the things that sets a good programmer apart is the willingness to fearlessly dig into someone else's code. I've heard it said before that you're respon

One of the things that sets a good programmer apart is the willingness to fearlessly dig into someone else's code. I've heard it said before that you're responsible for every line of code you ship to your users. It follows that you shouldn't treat your dependencies as black boxes. Getting a stack trace that's 3 or 4 levels deep in Django/React/<insert library here>? Dig under the covers and understand what's happening. Include that information in a bug report, and better yet, open a PR offering to fix it. You'll learn a whole lot and be more effective if you let your curiosity expand beyond code you/your team has written.

jotto 9y ago
No mention of time in the article or in your comment. With infinite time we could fix a lot of things.
- ef4 9y ago
  Sure, but I think many programmers underestimate the investment value of that time. Diving into codebases is a skill that gets stronger with use, such that you can eventually do it radically faster. That makes a much larger set of problems economically practical to fix.
- bsimpson 9y ago
  Of course you can expand this to be unreasonable - you could also make a strawman argument about how you need to understand all the compilers, OS APIs, and chip architectures that support your product. Obviously, that's not what I'm saying. I'm making a few assumptions: the code you're depending on was written by people like you (e.g. other open source volunteers with expertise in the same language as you); the limitation you're encountering is problematic for your users, product, or business; and that you'd rather have it work correctly than design around it (e.g. remove the affected feature). If those are all true, yeah, it's probably worth a week or two to figure out what's wrong and fix it (especially since you'll gain comfort, familiarity, and efficiency in that codebase over time - it might even inspire other enhancements). I realize that nobody has infinite time and I trust you to make reasonable decisions about what to spend where. Still, I implore you to not let yourselves be afraid of Other People's Code. It's probably not as scary as you think it is; after all, it was written by other people like you.
  - copperx 9y ago
    > It's probably not as scary as you think it is; after all, it was written by other people like you. Most interesting code has been written by people smarter than me (the kernel, compositing window manager, network stack, web rendering engine, virtual machine, etc.), and while reading that code probable isn't intractable, it would take me decades of effort to understand. At the same time, I think I can be an effective developer by building on top of those technologies (i.e., being a great plumber). And that's OK.
    - deleted 9y ago
      [deleted]
  - astura 9y ago
    >it's probably worth a week or two to figure out what's wrong and fix it Ha! I'm convinced that people who say stuff like this haven't worked where their paycheck comes from actual paying customers, as opposed to VC dollars or subsidies from other parts of the business's revenue. > Still, I implore you to not let yourselves be afraid of Other People's Code. Wtf? Nobody said they were and what is with the Condescending Proper Noun? >It's probably not as scary as you think it is Nobody said it was.
panic 9y ago
That's one thing that bothers me about modern Electron-style apps which depend on an entire web rendering engine -- actually digging through all the dependencies to track down a bug can be almost impossible. For example, I was hitting a bug in an Electron app a while back and wanted to track it down. I got the app building with a locally-built version of Electron, but it became clear the real bug involved Chromium's loader code. My poor 4-year-old laptop ran out of disk space before I could even download the whole libchromiumcontent repository, let alone build it!
- hinkley 9y ago
  I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. Every library demands a fraction of our attention greater than its relative fraction of the code. Complex calling conventions, deep call stacks. Why am I even using a single purpose library with multiple levels of abstraction in it? At this point a library should BE the abstraction, not contain them. I have to trace code through my code into yours into your dependencies. Just stop already. Keep It Simple Stupid.
  - gaius 9y ago
    I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. The other day I was debugging an app that - no joke - pulled in 600M of libraries that were all so it could use one file parsing function that could have been re-implemented in <100 lines. Crazy!
curtis 9y ago
> ... willingness to fearlessly dig into someone else's code. I've done this many times. The problem is that it takes a lot of time, and there's no way you can dig through more than a fraction of a large codebase. You gotta pick your battles.
- nathanaldensr 9y ago
  Pragmatism wins the day, for sure. I'll happily read someone else's code (e.g., an open-source GitHub repository I rely on) if it's necessary to troubleshoot issues or implement a feature. There needs to be a reason (i.e., motivation) to read code.
  - andrea_sdl 9y ago
    This is the only reason (at least for now) that makes me dig into a library code. And I always got some benefit from it. I think we often treat libraries as voodoo blackboxes, but they're more often thatn not created by normal people with great skills. Sometimes it's a great way to learn a way to code, or even better, to understand the behaviour of a library. I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know).
    - TeMPOraL 9y ago
      > I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know). Same here. It's about trivial inconveniences. I sometimes can be bothered to download and read through the codebase of a dependency, but most of the times I end up in third-party code happen when it's just a single jump-to-definition keypress away - the same keypress (M-.) which I use all the time for my code. That seamless browsing really blurs the difference.
    - WillReplyfFood 9y ago
      The blackbox vodoo view- sometimes is just a excuse for lazy evaluation. One can profile a black box- but many fear the moment they discover that there beloved "I know all the api calls from memory alone" library shows bad perfRomance.
tobr 9y ago
I always get an uneasy feeling when this is not (practically) possible. Whenever I add a dependency to a project I will at least skim through the source to get an idea of what it is I’m adding, and it pleases me a lot when a library is well written and self contained enough that this does not mostly leave me confused or overwhelmed. There’s a similar reason why I have such mixed feelings about some build tools and compilers, especially in the JS world - the output is often hideous and so unlike what a human might write. I know you can say that what the output looks like doesn’t really matter, as long as it works as it should. But it just makes it so much harder to feel confident that it actually does.
- icebraining 9y ago
  The output of gcc and clang probably don't look much like hand-written machine code either, though. Or, for that matter, the output of V8 and SpiderMonkey. You can't avoid the compilers :)
  - tobr 9y ago
    Of course! I’m not saying it’s rational. If I had experience hand-writing machine code maybe I would feel the same about that...
    - khedoros1 9y ago
      I've done bits of assembly, but I've also studied compilers. They felt less magical to me when I learned some of how they work. Then again, I'm primarily a C++ developer, so I'm used to the output of my build tools being non-trivial to understand.
  - deleted 9y ago
    [deleted]
- akvadrako 9y ago
  That's one reason rollup is better than webpack; the output looks almost hand rolled.
- 9y ago
taeric 9y ago
Don't let survivorship and confirmation bias cloud your judgment. What sets many successful people apart is not any one thing, but that they were successful. Many are willing to dive into other people's code in methods that are far deeper than I can comprehend, and yet they have failed at whatever task they were going about. So, yes. By and large, you shouldn't stop at your dependency boundaries. However, your job is to get results. For most of us, even if the dependency is bad, you are better off changing your code. Why? Because you can more quickly get a change in your codebase than you can in the dependency. And heaven help you if you think it is worth forking. This is not to say don't update the base place. By all means, please do. But don't wait for that change to land before you do something on your end, too. Even if that means not using your actual fix.
- icebraining 9y ago
  Forking is not that bad, unless upstream has a lot of activity yet takes a long time to accept patches. Many projects are mostly dormant, so keeping a patchset and occasionally rebasing is easy enough. We use git-aggregator[1] for easily applying pending PRs/MRs over the upstream branch, and for the most part, conflicts are rare. [1] https://pypi.python.org/pypi/git-aggregator https://pypi.python.org/pypi/git-aggregator
  - heavenlyblue 9y ago
    The thing I find with python/ruby ecosystems - is that the versions are generally pinned, so if you do fork a change for your specific need; you'll end up using it for the project. If the change isn't merged - then at least you may use it yourself.
    - taeric 9y ago
      These are both valid choices. I had not meant it to be that it is not possible. However, in both cases you now have a new line of work. It is common in my experience to find folks are behind in their first job, so adding a new one is dangerous.
- 9y ago
deleted 9y ago
[deleted]
natecavanaugh 9y ago
I can only say Amen to this. If you ship it, you either accept responsibility as far as you can, or you're "that" person who redirects you another department because it's not technically their job. Plus, as you mentioned the opportunity for learning is immense.
segphault 9y ago
> I've heard it said before that you're responsible for every line of code you ship to your users. This is a great way of thinking about it. I wish that more people took it to heart when choosing third-party dependencies to adopt, especially in the Node.js ecosystem where people are tragically cavalier about adding literally hundreds of under-scrutinized transitive dependencies to their projects.
geezerjay 9y ago
I agree with the spirit but in practice more often than not that's a complete waste of time and energy, unless you're trying to figure out the cause of a specific bug when you've already excluded your code as its source.
lawn 9y ago
To be sure. There's also a trade off to doing it too much as the time it can take to really go deep can effectively paralyze the development. You need to be able to properly use abstraction layers and be able to treat code as black boxes when needed, you only need sufficient knowledge to work with a system. Of course we shouldn't stop until we reach that level and too often developers give up. But it's also common, especially for very skilled or perfectionist developers, to go too far.
watwut 9y ago
Really good programmer is imo set apart by ability to decide when to dig into others code and when not.
chiefalchemist 9y ago
"...open a PR offering to fix it..." Perhaps I've been asking to wrong people (at the wrong time) - forgive me - but (as dar as I can tell) git has no easy / reasonable way to repo'tize a (vendor/) dependency. That means you have to bend over backwards (read: make too much extra effort) to patch such a library local and then eventually PR that change. This feels like a fairly significant flaw in the current OSS "ecosystem." It encourages the wrong behaviour, and discourages the right one. Or is there something I'm just not understanding?
- andrewaylett 9y ago
  It's not really so much that _git_ is lacking the functionality -- it's actually much more possible to do reasonable vendoring with git than with (say) subversion, as you can use submodules to point at your fork. To my mind the issue is more with tooling: you don't typically want to vendor your code, most of the time you want to use a published module. And the process for publishing your fork is too complex to be worthwhile. Also, the work to keep the fork itself up-to-date when you realise it's necessary is minimal, but by switching away from the original you lose tooling support for telling you when you need to make that update.
  - chiefalchemist 9y ago
    I understand there are workarounds, but they are just that. I mean, for example, I'm working away but hit a bug in a dependency. Now i have to fork, submodule, etc. I'm not expecting frictionless per se. But it just feels like there's more overhead than there needs to be. Also, it's possible (in rare cases) the vendor folder is not .gitignore'd. Now what? Submodules is out. Again, given the nature of most modern dev (i.e., dependencies) vit feels underprepared. Re: "...but by switching away from the original you lose tooling support for telling you when you need to make that update..." Yes. But doesn't this actually support the idea (mentioned above) not to treat your dependencies like black boxes? That is, no pain no gain? Yeah, perhaps some sort of simple notification your fork needs an update. Just the same, ALL code should, in theory, be treated as your own. You wouldn't blindly merge/commit a colleague's work, so why shouldn't your dependencies be semi thoroughly reviewed as well? Are we too busy to do the right thing(s)?
giancarlostoro 9y ago
I've only found time to do this with CherryPy really, mostly cause every time someone asks for help they say if something is missing in the documentation you could always read the source which is also commented and readable. It really is approachable and not too big. I never have time to go through the source of every single library but you raise a good time to do so: stack traces. Although when doing front-end JS code it's a bit less approachable if your files are already minified (client hands source like this) and the error becomes a lot less useful.
Daycrawler 9y ago
I just write that the library's function at hand is unreliable and can't be used this way.
Cthulhu_ 9y ago
It's why (in theory) I love Go; as opposed to the JS hell as mentioned by another commenter, it has a lot more emphasis on consistency and, in a way, predictability, so there's less surprises and distractions (e.g. code style) when you browse other people's code. Also no "black box" dependencies (usually), you pull every lib from source and have it on your machine.
noway421 9y ago
Isn't it the purpose of blackboxes, to abstract me from doing this? I'm here to get the work done and hopefully having to debug only code of my colleagues, why would I want to waste time on debugging something that should have been battle tested by myriads of people before me? Argument like this almost justifies leaky abstractions as something to be praised.
- amazingman 9y ago
  Isn’t the purpose of open source also to empower you, the developer using the library, to fix the bug you find so that everyone may benefit?
bitL 9y ago
> willingness to fearlessly dig into someone else's code. That's kinda illusion. If you start digging into some really complex code, like distributed processing core, meta-programming framework, low-level optimizations, it can take you years to understand all that is going on. Often it's more economical just to rewrite it, even if the previous system was far better than what you'll produce. Whether companies like it or not, projects live with original team members and once they move on, it's getting complicated to maintain or extend them.
- nevir 9y ago
  The inherent danger in rewriting dependencies always seems to be greatly underestimated, though. Especially when this is your first time tackling the problem space in earnest. * Getting the solution 80% of the way there is often pretty fast (which ends up deceiving everyone depending on it "it's almost done!") * You are nearly guaranteed to encounter a bunch of bugs that were already solved in the original dependency. * Due to the previous points, there's a big chance that your initial design had a few fatal flaws, and requires significant refactoring to address (making the project take longer, and if not careful making it much harder to understand & maintain) Rewriting dependencies can definitely be worth it from a code ownership/maintenance point of view; but more often than not, you'll end up creating a cycle where the new version of the dependency is just as obtuse as the previous—and once you leave, the same problem all over again. If a dependency is well documented, has a clear architecture, and follows consistent coding practices, it will stand the test of time (including loss of authors/maintainers). Most people/companies aren't willing to put the time/effort into that level of polish.
shortoncash 9y ago
Take all my upvotes and offer a newsletter for subscription please. The lack of digging is a frustration I have with a lot of new guys who join the team. For example, they might create something new and tack on some new, untested piece of code without reading whether something already exists within the same codebase. Even in a startup environment where documentation is scarce, it's worthwhile to ask the question of, "Did someone else here think of this problem here before me? What did they do?"
- mberger 9y ago
  It comes down to unknown unknowns. Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. The second problem is that people will keep stuff hidden. Have that sweet script that creates 1000 users for testing? You might share it with the team, you might also think its too trivial to bother sharing. That being said, I can see how it can be frustrating. We have packages upon packages of existing functionality, languishing somewhere in source control, used maybe only once. In the meantime, someone is going through the cycle, not only of ignoring what someone put on gitub, but what has already been tested and qa'd by their own company.
  - clintonb 9y ago
    > Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. It helps to have a company culture that encourages openness and asking questions. My current company goes so far as to have a Slack channel, #askanything, where folks can ask anything. We'd much rather you ask early in the development process than pursue a path that will lead to duplication.
chapill 9y ago
>Include that information in a bug report, and better yet, open a PR offering to fix it. In my experience, never happens on permissively licensed projects. "Contribute time and code to open source on company time? Are you mad? Fork it and keep that as our competitive advantage!" Nevermind how dumb this approach actually is, it's what management thinks. "GPL? Oh, we have to? Oh, well, I guess if we have to."
yorby 9y ago
Do you consider the OS a "library"? Who checks the OS's code before shipping a program? What about the programming language's code? Do you code in assembly to reduce the amount of dependencies?

jotto 9y ago

No mention of time in the article or in your comment. With infinite time we could fix a lot of things.

ef4 9y ago
Sure, but I think many programmers underestimate the investment value of that time. Diving into codebases is a skill that gets stronger with use, such that you can eventually do it radically faster. That makes a much larger set of problems economically practical to fix.
bsimpson 9y ago
Of course you can expand this to be unreasonable - you could also make a strawman argument about how you need to understand all the compilers, OS APIs, and chip architectures that support your product. Obviously, that's not what I'm saying. I'm making a few assumptions: the code you're depending on was written by people like you (e.g. other open source volunteers with expertise in the same language as you); the limitation you're encountering is problematic for your users, product, or business; and that you'd rather have it work correctly than design around it (e.g. remove the affected feature). If those are all true, yeah, it's probably worth a week or two to figure out what's wrong and fix it (especially since you'll gain comfort, familiarity, and efficiency in that codebase over time - it might even inspire other enhancements). I realize that nobody has infinite time and I trust you to make reasonable decisions about what to spend where. Still, I implore you to not let yourselves be afraid of Other People's Code. It's probably not as scary as you think it is; after all, it was written by other people like you.
- copperx 9y ago
  > It's probably not as scary as you think it is; after all, it was written by other people like you. Most interesting code has been written by people smarter than me (the kernel, compositing window manager, network stack, web rendering engine, virtual machine, etc.), and while reading that code probable isn't intractable, it would take me decades of effort to understand. At the same time, I think I can be an effective developer by building on top of those technologies (i.e., being a great plumber). And that's OK.
  - deleted 9y ago
    [deleted]
- astura 9y ago
  >it's probably worth a week or two to figure out what's wrong and fix it Ha! I'm convinced that people who say stuff like this haven't worked where their paycheck comes from actual paying customers, as opposed to VC dollars or subsidies from other parts of the business's revenue. > Still, I implore you to not let yourselves be afraid of Other People's Code. Wtf? Nobody said they were and what is with the Condescending Proper Noun? >It's probably not as scary as you think it is Nobody said it was.

panic 9y ago

That's one thing that bothers me about modern Electron-style apps which depend on an entire web rendering engine -- actually digging through all the dependencies to track down a bug can be almost impossible. For example, I was hitting a bug in an Electron app a while back and wanted to track it down. I got the app building with a locally-built version of Electron, but it became clear the real bug involved Chromium's loader code. My poor 4-year-old laptop ran out of disk space before I could even download the whole libchromiumcontent repository, let alone build it!

hinkley 9y ago
I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. Every library demands a fraction of our attention greater than its relative fraction of the code. Complex calling conventions, deep call stacks. Why am I even using a single purpose library with multiple levels of abstraction in it? At this point a library should BE the abstraction, not contain them. I have to trace code through my code into yours into your dependencies. Just stop already. Keep It Simple Stupid.
- gaius 9y ago
  I don’t think Library designers have adapted yet to a library heavy world. We still write them like we are using dozens but we have hundreds. The other day I was debugging an app that - no joke - pulled in 600M of libraries that were all so it could use one file parsing function that could have been re-implemented in <100 lines. Crazy!

curtis 9y ago

> ... willingness to fearlessly dig into someone else's code. I've done this many times. The problem is that it takes a lot of time, and there's no way you can dig through more than a fraction of a large codebase. You gotta pick your battles.

nathanaldensr 9y ago
Pragmatism wins the day, for sure. I'll happily read someone else's code (e.g., an open-source GitHub repository I rely on) if it's necessary to troubleshoot issues or implement a feature. There needs to be a reason (i.e., motivation) to read code.
- andrea_sdl 9y ago
  This is the only reason (at least for now) that makes me dig into a library code. And I always got some benefit from it. I think we often treat libraries as voodoo blackboxes, but they're more often thatn not created by normal people with great skills. Sometimes it's a great way to learn a way to code, or even better, to understand the behaviour of a library. I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know).
  - TeMPOraL 9y ago
    > I found myself doing it more often when the IDE itself downloaded and linked the sources. (Lazy, I know). Same here. It's about trivial inconveniences. I sometimes can be bothered to download and read through the codebase of a dependency, but most of the times I end up in third-party code happen when it's just a single jump-to-definition keypress away - the same keypress (M-.) which I use all the time for my code. That seamless browsing really blurs the difference.
  - WillReplyfFood 9y ago
    The blackbox vodoo view- sometimes is just a excuse for lazy evaluation. One can profile a black box- but many fear the moment they discover that there beloved "I know all the api calls from memory alone" library shows bad perfRomance.

tobr 9y ago

I always get an uneasy feeling when this is not (practically) possible. Whenever I add a dependency to a project I will at least skim through the source to get an idea of what it is I’m adding, and it pleases me a lot when a library is well written and self contained enough that this does not mostly leave me confused or overwhelmed. There’s a similar reason why I have such mixed feelings about some build tools and compilers, especially in the JS world - the output is often hideous and so unlike what a human might write. I know you can say that what the output looks like doesn’t really matter, as long as it works as it should. But it just makes it so much harder to feel confident that it actually does.

icebraining 9y ago
The output of gcc and clang probably don't look much like hand-written machine code either, though. Or, for that matter, the output of V8 and SpiderMonkey. You can't avoid the compilers :)
- tobr 9y ago
  Of course! I’m not saying it’s rational. If I had experience hand-writing machine code maybe I would feel the same about that...
  - khedoros1 9y ago
    I've done bits of assembly, but I've also studied compilers. They felt less magical to me when I learned some of how they work. Then again, I'm primarily a C++ developer, so I'm used to the output of my build tools being non-trivial to understand.
- deleted 9y ago
  [deleted]
akvadrako 9y ago
That's one reason rollup is better than webpack; the output looks almost hand rolled.
9y ago

taeric 9y ago

Don't let survivorship and confirmation bias cloud your judgment. What sets many successful people apart is not any one thing, but that they were successful. Many are willing to dive into other people's code in methods that are far deeper than I can comprehend, and yet they have failed at whatever task they were going about. So, yes. By and large, you shouldn't stop at your dependency boundaries. However, your job is to get results. For most of us, even if the dependency is bad, you are better off changing your code. Why? Because you can more quickly get a change in your codebase than you can in the dependency. And heaven help you if you think it is worth forking. This is not to say don't update the base place. By all means, please do. But don't wait for that change to land before you do something on your end, too. Even if that means not using your actual fix.

icebraining 9y ago
Forking is not that bad, unless upstream has a lot of activity yet takes a long time to accept patches. Many projects are mostly dormant, so keeping a patchset and occasionally rebasing is easy enough. We use git-aggregator[1] for easily applying pending PRs/MRs over the upstream branch, and for the most part, conflicts are rare. [1] https://pypi.python.org/pypi/git-aggregator https://pypi.python.org/pypi/git-aggregator
- heavenlyblue 9y ago
  The thing I find with python/ruby ecosystems - is that the versions are generally pinned, so if you do fork a change for your specific need; you'll end up using it for the project. If the change isn't merged - then at least you may use it yourself.
  - taeric 9y ago
    These are both valid choices. I had not meant it to be that it is not possible. However, in both cases you now have a new line of work. It is common in my experience to find folks are behind in their first job, so adding a new one is dangerous.
9y ago

deleted 9y ago

[deleted]

natecavanaugh 9y ago

I can only say Amen to this. If you ship it, you either accept responsibility as far as you can, or you're "that" person who redirects you another department because it's not technically their job. Plus, as you mentioned the opportunity for learning is immense.

segphault 9y ago

> I've heard it said before that you're responsible for every line of code you ship to your users. This is a great way of thinking about it. I wish that more people took it to heart when choosing third-party dependencies to adopt, especially in the Node.js ecosystem where people are tragically cavalier about adding literally hundreds of under-scrutinized transitive dependencies to their projects.

geezerjay 9y ago

I agree with the spirit but in practice more often than not that's a complete waste of time and energy, unless you're trying to figure out the cause of a specific bug when you've already excluded your code as its source.

lawn 9y ago

To be sure. There's also a trade off to doing it too much as the time it can take to really go deep can effectively paralyze the development. You need to be able to properly use abstraction layers and be able to treat code as black boxes when needed, you only need sufficient knowledge to work with a system. Of course we shouldn't stop until we reach that level and too often developers give up. But it's also common, especially for very skilled or perfectionist developers, to go too far.

watwut 9y ago

Really good programmer is imo set apart by ability to decide when to dig into others code and when not.

chiefalchemist 9y ago

"...open a PR offering to fix it..." Perhaps I've been asking to wrong people (at the wrong time) - forgive me - but (as dar as I can tell) git has no easy / reasonable way to repo'tize a (vendor/) dependency. That means you have to bend over backwards (read: make too much extra effort) to patch such a library local and then eventually PR that change. This feels like a fairly significant flaw in the current OSS "ecosystem." It encourages the wrong behaviour, and discourages the right one. Or is there something I'm just not understanding?

andrewaylett 9y ago
It's not really so much that _git_ is lacking the functionality -- it's actually much more possible to do reasonable vendoring with git than with (say) subversion, as you can use submodules to point at your fork. To my mind the issue is more with tooling: you don't typically want to vendor your code, most of the time you want to use a published module. And the process for publishing your fork is too complex to be worthwhile. Also, the work to keep the fork itself up-to-date when you realise it's necessary is minimal, but by switching away from the original you lose tooling support for telling you when you need to make that update.
- chiefalchemist 9y ago
  I understand there are workarounds, but they are just that. I mean, for example, I'm working away but hit a bug in a dependency. Now i have to fork, submodule, etc. I'm not expecting frictionless per se. But it just feels like there's more overhead than there needs to be. Also, it's possible (in rare cases) the vendor folder is not .gitignore'd. Now what? Submodules is out. Again, given the nature of most modern dev (i.e., dependencies) vit feels underprepared. Re: "...but by switching away from the original you lose tooling support for telling you when you need to make that update..." Yes. But doesn't this actually support the idea (mentioned above) not to treat your dependencies like black boxes? That is, no pain no gain? Yeah, perhaps some sort of simple notification your fork needs an update. Just the same, ALL code should, in theory, be treated as your own. You wouldn't blindly merge/commit a colleague's work, so why shouldn't your dependencies be semi thoroughly reviewed as well? Are we too busy to do the right thing(s)?

giancarlostoro 9y ago

I've only found time to do this with CherryPy really, mostly cause every time someone asks for help they say if something is missing in the documentation you could always read the source which is also commented and readable. It really is approachable and not too big. I never have time to go through the source of every single library but you raise a good time to do so: stack traces. Although when doing front-end JS code it's a bit less approachable if your files are already minified (client hands source like this) and the error becomes a lot less useful.

Daycrawler 9y ago

I just write that the library's function at hand is unreliable and can't be used this way.

Cthulhu_ 9y ago

It's why (in theory) I love Go; as opposed to the JS hell as mentioned by another commenter, it has a lot more emphasis on consistency and, in a way, predictability, so there's less surprises and distractions (e.g. code style) when you browse other people's code. Also no "black box" dependencies (usually), you pull every lib from source and have it on your machine.

noway421 9y ago

Isn't it the purpose of blackboxes, to abstract me from doing this? I'm here to get the work done and hopefully having to debug only code of my colleagues, why would I want to waste time on debugging something that should have been battle tested by myriads of people before me? Argument like this almost justifies leaky abstractions as something to be praised.

amazingman 9y ago
Isn’t the purpose of open source also to empower you, the developer using the library, to fix the bug you find so that everyone may benefit?

bitL 9y ago

> willingness to fearlessly dig into someone else's code. That's kinda illusion. If you start digging into some really complex code, like distributed processing core, meta-programming framework, low-level optimizations, it can take you years to understand all that is going on. Often it's more economical just to rewrite it, even if the previous system was far better than what you'll produce. Whether companies like it or not, projects live with original team members and once they move on, it's getting complicated to maintain or extend them.

nevir 9y ago
The inherent danger in rewriting dependencies always seems to be greatly underestimated, though. Especially when this is your first time tackling the problem space in earnest. * Getting the solution 80% of the way there is often pretty fast (which ends up deceiving everyone depending on it "it's almost done!") * You are nearly guaranteed to encounter a bunch of bugs that were already solved in the original dependency. * Due to the previous points, there's a big chance that your initial design had a few fatal flaws, and requires significant refactoring to address (making the project take longer, and if not careful making it much harder to understand & maintain) Rewriting dependencies can definitely be worth it from a code ownership/maintenance point of view; but more often than not, you'll end up creating a cycle where the new version of the dependency is just as obtuse as the previous—and once you leave, the same problem all over again. If a dependency is well documented, has a clear architecture, and follows consistent coding practices, it will stand the test of time (including loss of authors/maintainers). Most people/companies aren't willing to put the time/effort into that level of polish.

shortoncash 9y ago

Take all my upvotes and offer a newsletter for subscription please. The lack of digging is a frustration I have with a lot of new guys who join the team. For example, they might create something new and tack on some new, untested piece of code without reading whether something already exists within the same codebase. Even in a startup environment where documentation is scarce, it's worthwhile to ask the question of, "Did someone else here think of this problem here before me? What did they do?"

mberger 9y ago
It comes down to unknown unknowns. Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. The second problem is that people will keep stuff hidden. Have that sweet script that creates 1000 users for testing? You might share it with the team, you might also think its too trivial to bother sharing. That being said, I can see how it can be frustrating. We have packages upon packages of existing functionality, languishing somewhere in source control, used maybe only once. In the meantime, someone is going through the cycle, not only of ignoring what someone put on gitub, but what has already been tested and qa'd by their own company.
- clintonb 9y ago
  > Should I poll my supervisor/co-worker about every problem to see if there is already a solution? As the new guy, I have no idea who or what I should poll people about. It helps to have a company culture that encourages openness and asking questions. My current company goes so far as to have a Slack channel, #askanything, where folks can ask anything. We'd much rather you ask early in the development process than pursue a path that will lead to duplication.

chapill 9y ago

>Include that information in a bug report, and better yet, open a PR offering to fix it. In my experience, never happens on permissively licensed projects. "Contribute time and code to open source on company time? Are you mad? Fork it and keep that as our competitive advantage!" Nevermind how dumb this approach actually is, it's what management thinks. "GPL? Oh, we have to? Oh, well, I guess if we have to."

yorby 9y ago

Do you consider the OS a "library"? Who checks the OS's code before shipping a program? What about the programming language's code? Do you code in assembly to reduce the amount of dependencies?