9 ms·

Here's an overlooked one: your dependencies. Every programmer should read the (presumably open source) code they depend on, but almost nobody does it. Some loo

Here's an overlooked one: your dependencies. Every programmer should read the (presumably open source) code they depend on, but almost nobody does it. Some look at documentation, but not everyone even does that. What you may find is that some of the code you depend on is garbage, and may even motivate improvements. This may be less feasible for giant messes such as most front-end tooling and frameworks that exist today.

pc86 9y ago
Great advice, especially if the improvements are made to the open source package(s) so that everyone can benefit. Proprietary forks/upgrades to open source packages has a much narrower impact.
aczerepinski 9y ago
I read my dependency code partially as a way to learn the language. Several times in the past year I've been assigned to a project writing a language I don't know, and the major dependencies have often been a great example of well written, idiomatic code in that language.
ATsch 9y ago
I can really recommend this. I've made a habit of just casually browsing through the source of dependencies and just opening up anything that sounds interesting.
seanwilson 9y ago
> Every programmer should read the (presumably open source) code they depend on, but almost nobody does it. Has this ever been beneficial? I generally just check if a library is popular and maintained and that's always been enough for me. If the code really is horrendous nobody is going to be using it and if you always read the hundred of thousands of lines of code you depend on you'd never get anything done.
- gwbas1c 9y ago
  Have you ever used a library and thought, "ugh, it should be written THIS way." When you review a dependency, you answer why it should not be written THIS way. (Or why it's junk and shouldn't be used.)
- daliwali 9y ago
  Popularity is not an indicator of anything other than popularity. McDonalds is the most popular burger sold in the world, that does not make it good. Length is an important consideration and most never consider it, partly why software is notorious for getting bloated over time. At some point it would be easier to rewrite code from scratch with less complexity and overhead.
  - kristianc 9y ago
    > Popularity is not an indicator of anything other than popularity. McDonalds is the most popular burger sold in the world, that does not make it good. It doesn't make it good, but it does mean that it's unlikely to make you ill.
  - haburka 9y ago
    Popularity means more eyes have looked at it, more issues have been made, more contributors exist. Of course, I've had issues with dependencies that were popular but generally it holds that a library that's been around for a while and still finding usage means that it does a good job at it. The difference between code and a burger is that anyone can consume a burger but code is consumed almost exclusively by professionals. A better comparison would be power tools where you'll often find that if a brand is popular, it tends to make solid, long lasting products.
    - daliwali 9y ago
      The point was more about popularity having no correlation with anything else. There is tons of unpopular code that is really, really good, these are hidden gems. Likewise, there is popular code that is quite bad or mediocre regardless of number of contributors. To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good.
      - seanwilson 9y ago
        > Likewise, there is popular code that is quite bad or mediocre regardless of number of contributors. Like what? If you're looking for a JavaScript library for example, you can't really go wrong picking one that has many contributors, an active issue tracker, thousands of stars, lots of forks etc. Unless it's some core part of your stack, I don't see how it's practical to spend a lot of time hunting down super well implemented libraries that aren't likely to be supported in the future because nobody uses them. > To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good. I'm not a huge fan of PHP but you can still write good software in it. Libraries are relatively easy to swap out compared to say operating systems and programming languages. Personally I find many of the best libraries tend to be the most popular ones.
- cosmosgenius 9y ago
  It has been immensely beneficial. Reading the underlining framework probably made me write less code than i would have otherwise(plus the insights). The best example in my case was to extend the classes of Django REST Framework.
- wmsiler 9y ago
  I'm sure they weren't recommending you read every line and grok the whole code base of each of your dependencies. I don't know how helpful reading source is for determining whether or not to use a dependency, but I find it very helpful to have at least a small understanding of how my dependencies work: 1. I find it useful for debugging. If a coworker or I have used a library incorrectly, understanding how the library works can help us figure out why it isn't doing what we wanted. Also, my IDE lets me do step-through debugging through dependency code as well, which can be helpful. 2. It helps with determining the capabilities of a library. If I know how a library works, then I can make reasonable guesses about what features the library has (or could easily add). I find myself thinking, "I'd like to do X, and from what I know of how library foo works, that seems like something it should be able to do." So I can go investigate that. 3. It helps with knowing the performance characteristics of the library. Just yesterday, I needed to implement some stats logic in java. I found an Apache Commons class that does exactly what I need, but by looking at the source code, I found that it uses an O(n) algorithm, which is fine for the cases they are probably targeting. In this case, I need more performance than that, and it's not hard to write this as O(log n). So I wrote my own instead. If my IDE (Intellij) didn't give me super easy access to dependency source code, this would have been more painful.
  - fiddlerwoaroof 9y ago
    #2 is sort of a mixed bag, though, reading the source is great for finding out what it can do, but it also opens up to depending on implementation details rather than the library's specified behavior.
- dom0 9y ago
  Reviewing dependencies avoids dragging shitty, obviously broken and/or horrifically inefficient code around. You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Or a crypto library that documents almost nothing and requires you to review x86 assembler to find how parameters are actually interpreted cough? Taking at least a cursory glance at the code is one part of vetting dependencies before using them. Other activities would be: checking use of best practices ("Your parser written in K&R C is probably fine without fuzzing and any tests"), availability of documentation ("source is there, no?"), checking for known vulnerabilities and bugs (First open issue: "Library segfaults when parsing long lines of []{}"), maintainership ("Last commit: 2004") and portability ("(void*)a + (int)5.7").
  - Animats 9y ago
    You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Python's CPickle comes to mind. There's a race condition in there somewhere, but I've never been able to reproduce it with a small program. [1] Using the version of Pickle written in Python works. This sort of thing is why I'm somewhat negative on the argument that "if Python is too slow, write the important parts in C". It's too easy to break Python's memory model in C code. [1] http://bugs.python.org/issue23655 http://bugs.python.org/issue23655
    - dom0 9y ago
      On the other hand it's fairly easy to not break Python's memory model if you don't unlock the GIL :) Edit: Ah that kind of bug can be ... tricky.
  - seanwilson 9y ago
    > You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Or a crypto library that documents almost nothing and requires you to review x86 assembler to find how parameters are actually interpreted cough? If it corrupts data and has terrible documentation it wouldn't be popular. It's usually very obvious from the community around the library if it's got major problems or not.
- ryanlol 9y ago
  >Has this ever been beneficial? If you want to write secure code it's necessary. Otherwise you just end up with PHP web apps that have remotely exploitable memory corruption bugs because the developer didn't understand that he absolutely has to sanitize all user inputs to certain functions. >If the code really is horrendous nobody is going to be using it This just makes it pretty clear that you haven't ever looked.
  - seanwilson 9y ago
    Maybe if you're working for something that requires very high security but it's completely unrealistic that you can security audit all your dependencies. I'm pretty confident Angular, React, Vue etc. don't have any glaringly obvious security problems that all the other contributors missed for example. Do you check the source code for Apache, MySQL, PHP etc.? If I'm picking a library for a component where its flaws would be high impact (e.g. security) and there isn't a strong community behind that library I'm going to be very cautious though.
- deleted 9y ago
  [deleted]
MattConfluence 9y ago
I program a lot of Python using PyCharm, and one of my favorite features of this combination of language and IDE is how easy it is to hop into the source of your dependencies. Anything you have imported into your code you can bring up the context menu for and click "go to declaration" (or use the relevant keyboard shortcut). Python dependencies are source code, not compiled blobs, and the import syntax makes it very explicit where definitions come from. If I'm unsure how to do something with the library I usually look at the source of the relevant module before looking up online docs.
- reitanqild 9y ago
  Java, Maven and Netbeans is almost as easy but since dependencies are compiled I have to click one more time, -download sources- and there I have it.
  - jdmichal 9y ago
    Eclipse will automatically attempt to download sources from Maven when you jump into a dependency.
    - reitanqild 9y ago
      I suspected it was easy in eclipse as well and should possibly have mentioned that. That said let me add that IntelliJ probably has similar systems.
      - nazgul17 9y ago
        If you don't have the source, IntelliJ will decompile the binary and I must say it does a terrific job. It also handles downloads via Maven, but I have only done it only once and don't remember the details.
- Entalpi 9y ago
  This is why I love C++ and C even though you cant find the source you can read the headers.
noxToken 9y ago
Paraphrased from a professor when he substituted my intro programming course: Code is a valuable gemstone covered in layers of shit. Always aim to create diamonds, not CZ, and keep at least one face of that diamond shit free. I always thought was a very pessimistic way to view code until I landed my first job. Algorithm exercises and toy projects for capstone can be shit free diamonds every time. Working with finicky clients, legacy code bases, and peers who don't care about quality quickly proved that sometimes you're destined to make topaz double-dipped in dung. Dependencies are no exception to this. I've worked with a mapper that works pretty well out of the box. I ran into an issue with how Strings were mapped to boolean values. It turned out the default mapping values for those conversions were yes, no, y, n, and possibly some other variation of those String values. It didn't include true and false by default.
- nstart 9y ago
  I had a fun one recently. Just the other day in fact. The elastic beanstalk cli from aws was refusing to accept my branches when I tried to refer to a code push from a codecommit repo. Turns out branches with / in it can't be used because it conflicts with the way the cli parses its parameters. Basically it expects a parameter along the lines of codecommit/RepoName/branchName And then it splits the string and if there are more than 3 items as a result it returns from the function saying invalid value. I had to correct it to use it myself but I found it fascinating since it suddenly made the somewhat mystical innards of aws seem a little more human. At least that's how I felt :D.
  - pyre 9y ago
    I would submit a bug report. I've reported bugs in aws cli tools before, and they were patched.
    - taormina 9y ago
      They also accept PRs sometimes.
jschwartzi 9y ago
This is especially necessary for driver code from chip manufacturers. Most of that stuff is written to provide demonstrations and test the part, and it should not be trusted.
mixedbit 9y ago
Just a brief study (~few hours) of mysql and postgresql sources allowed me to made a definite decision which of the two is a better choice for all my web projects.
- brianwawok 9y ago
  Do tell ")
- lufte 9y ago
  Please elaborate. Did you write a blog post or something like that to document the experience?
- agentgt 9y ago
  Seriously? I am assuming you are not a database expert but basing this on the look and organization of the code and not the data structures and design. If that is not true than please share your findings instead of teasing us. I'm curious would you do the same with your editor (or IDE)? You should see the source code for those behemoths (with the exception of the rewrites like nvim) they are pretty nasty. Are you going to pick a different editor just because Vim or Emacs (or whatever editor) has nasty code? Guess what IDE has pretty clean code and is extremely modularized... Eclipse. I can tell you Eclipse is not a comfortable editor (and I use it all the time because I'm used to it not because the code is elegant). Mature projects often have pretty ugly code but they are mature and stable. Ugly code also doesn't necessarily mean poor design (e.g. if your editor is single threaded and has nice looking code its still single threaded).
  - samastur 9y ago
    There's a difference of category between your and parents example. Editors and IDEs will not make your code crash when it is deployed no matter how flaky or not they are. Databases on the other hand can. I wouldn't imagine it being controversial to prefer well engineered code over less so for bits you will need to depend on.
    - 9y ago
TearsInTheRain 9y ago
You depend on so much code, how is this even remotely feasible?
- daliwali 9y ago
  This is expected in some fields such as embedded programming, it's reasonable to look at source code just to figure out how hardware works. Or web front-end where you shouldn't be sending huge payloads to the browser anyways. You don't always need to look at the lowest level, just direct dependencies.
lsiebert 9y ago
Crappy documentation for dynamically typed code has often made me read my dependencies. Statically typed languages are much more likely to be self documenting, though occasionally, with Golang for example, they will return a structure with the same name as another structure in the program, and it's hard to figure out which of these is being returned.

pc86 9y ago

Great advice, especially if the improvements are made to the open source package(s) so that everyone can benefit. Proprietary forks/upgrades to open source packages has a much narrower impact.

aczerepinski 9y ago

I read my dependency code partially as a way to learn the language. Several times in the past year I've been assigned to a project writing a language I don't know, and the major dependencies have often been a great example of well written, idiomatic code in that language.

ATsch 9y ago

I can really recommend this. I've made a habit of just casually browsing through the source of dependencies and just opening up anything that sounds interesting.

seanwilson 9y ago

> Every programmer should read the (presumably open source) code they depend on, but almost nobody does it. Has this ever been beneficial? I generally just check if a library is popular and maintained and that's always been enough for me. If the code really is horrendous nobody is going to be using it and if you always read the hundred of thousands of lines of code you depend on you'd never get anything done.

gwbas1c 9y ago
Have you ever used a library and thought, "ugh, it should be written THIS way." When you review a dependency, you answer why it should not be written THIS way. (Or why it's junk and shouldn't be used.)
daliwali 9y ago
Popularity is not an indicator of anything other than popularity. McDonalds is the most popular burger sold in the world, that does not make it good. Length is an important consideration and most never consider it, partly why software is notorious for getting bloated over time. At some point it would be easier to rewrite code from scratch with less complexity and overhead.
- kristianc 9y ago
  > Popularity is not an indicator of anything other than popularity. McDonalds is the most popular burger sold in the world, that does not make it good. It doesn't make it good, but it does mean that it's unlikely to make you ill.
- haburka 9y ago
  Popularity means more eyes have looked at it, more issues have been made, more contributors exist. Of course, I've had issues with dependencies that were popular but generally it holds that a library that's been around for a while and still finding usage means that it does a good job at it. The difference between code and a burger is that anyone can consume a burger but code is consumed almost exclusively by professionals. A better comparison would be power tools where you'll often find that if a brand is popular, it tends to make solid, long lasting products.
  - daliwali 9y ago
    The point was more about popularity having no correlation with anything else. There is tons of unpopular code that is really, really good, these are hidden gems. Likewise, there is popular code that is quite bad or mediocre regardless of number of contributors. To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good.
    - seanwilson 9y ago
      > Likewise, there is popular code that is quite bad or mediocre regardless of number of contributors. Like what? If you're looking for a JavaScript library for example, you can't really go wrong picking one that has many contributors, an active issue tracker, thousands of stars, lots of forks etc. Unless it's some core part of your stack, I don't see how it's practical to spend a lot of time hunting down super well implemented libraries that aren't likely to be supported in the future because nobody uses them. > To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good. I'm not a huge fan of PHP but you can still write good software in it. Libraries are relatively easy to swap out compared to say operating systems and programming languages. Personally I find many of the best libraries tend to be the most popular ones.
cosmosgenius 9y ago
It has been immensely beneficial. Reading the underlining framework probably made me write less code than i would have otherwise(plus the insights). The best example in my case was to extend the classes of Django REST Framework.
wmsiler 9y ago
I'm sure they weren't recommending you read every line and grok the whole code base of each of your dependencies. I don't know how helpful reading source is for determining whether or not to use a dependency, but I find it very helpful to have at least a small understanding of how my dependencies work: 1. I find it useful for debugging. If a coworker or I have used a library incorrectly, understanding how the library works can help us figure out why it isn't doing what we wanted. Also, my IDE lets me do step-through debugging through dependency code as well, which can be helpful. 2. It helps with determining the capabilities of a library. If I know how a library works, then I can make reasonable guesses about what features the library has (or could easily add). I find myself thinking, "I'd like to do X, and from what I know of how library foo works, that seems like something it should be able to do." So I can go investigate that. 3. It helps with knowing the performance characteristics of the library. Just yesterday, I needed to implement some stats logic in java. I found an Apache Commons class that does exactly what I need, but by looking at the source code, I found that it uses an O(n) algorithm, which is fine for the cases they are probably targeting. In this case, I need more performance than that, and it's not hard to write this as O(log n). So I wrote my own instead. If my IDE (Intellij) didn't give me super easy access to dependency source code, this would have been more painful.
- fiddlerwoaroof 9y ago
  #2 is sort of a mixed bag, though, reading the source is great for finding out what it can do, but it also opens up to depending on implementation details rather than the library's specified behavior.
dom0 9y ago
Reviewing dependencies avoids dragging shitty, obviously broken and/or horrifically inefficient code around. You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Or a crypto library that documents almost nothing and requires you to review x86 assembler to find how parameters are actually interpreted cough? Taking at least a cursory glance at the code is one part of vetting dependencies before using them. Other activities would be: checking use of best practices ("Your parser written in K&R C is probably fine without fuzzing and any tests"), availability of documentation ("source is there, no?"), checking for known vulnerabilities and bugs (First open issue: "Library segfaults when parsing long lines of []{}"), maintainership ("Last commit: 2004") and portability ("(void*)a + (int)5.7").
- Animats 9y ago
  You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Python's CPickle comes to mind. There's a race condition in there somewhere, but I've never been able to reproduce it with a small program. [1] Using the version of Pickle written in Python works. This sort of thing is why I'm somewhat negative on the argument that "if Python is too slow, write the important parts in C". It's too easy to break Python's memory model in C code. [1] http://bugs.python.org/issue23655 http://bugs.python.org/issue23655
  - dom0 9y ago
    On the other hand it's fairly easy to not break Python's memory model if you don't unlock the GIL :) Edit: Ah that kind of bug can be ... tricky.
- seanwilson 9y ago
  > You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Or a crypto library that documents almost nothing and requires you to review x86 assembler to find how parameters are actually interpreted cough? If it corrupts data and has terrible documentation it wouldn't be popular. It's usually very obvious from the community around the library if it's got major problems or not.
ryanlol 9y ago
>Has this ever been beneficial? If you want to write secure code it's necessary. Otherwise you just end up with PHP web apps that have remotely exploitable memory corruption bugs because the developer didn't understand that he absolutely has to sanitize all user inputs to certain functions. >If the code really is horrendous nobody is going to be using it This just makes it pretty clear that you haven't ever looked.
- seanwilson 9y ago
  Maybe if you're working for something that requires very high security but it's completely unrealistic that you can security audit all your dependencies. I'm pretty confident Angular, React, Vue etc. don't have any glaringly obvious security problems that all the other contributors missed for example. Do you check the source code for Apache, MySQL, PHP etc.? If I'm picking a library for a component where its flaws would be high impact (e.g. security) and there isn't a strong community behind that library I'm going to be very cautious though.
deleted 9y ago
[deleted]

MattConfluence 9y ago

I program a lot of Python using PyCharm, and one of my favorite features of this combination of language and IDE is how easy it is to hop into the source of your dependencies. Anything you have imported into your code you can bring up the context menu for and click "go to declaration" (or use the relevant keyboard shortcut). Python dependencies are source code, not compiled blobs, and the import syntax makes it very explicit where definitions come from. If I'm unsure how to do something with the library I usually look at the source of the relevant module before looking up online docs.

reitanqild 9y ago
Java, Maven and Netbeans is almost as easy but since dependencies are compiled I have to click one more time, -download sources- and there I have it.
- jdmichal 9y ago
  Eclipse will automatically attempt to download sources from Maven when you jump into a dependency.
  - reitanqild 9y ago
    I suspected it was easy in eclipse as well and should possibly have mentioned that. That said let me add that IntelliJ probably has similar systems.
    - nazgul17 9y ago
      If you don't have the source, IntelliJ will decompile the binary and I must say it does a terrific job. It also handles downloads via Maven, but I have only done it only once and don't remember the details.
Entalpi 9y ago
This is why I love C++ and C even though you cant find the source you can read the headers.

noxToken 9y ago

Paraphrased from a professor when he substituted my intro programming course: Code is a valuable gemstone covered in layers of shit. Always aim to create diamonds, not CZ, and keep at least one face of that diamond shit free. I always thought was a very pessimistic way to view code until I landed my first job. Algorithm exercises and toy projects for capstone can be shit free diamonds every time. Working with finicky clients, legacy code bases, and peers who don't care about quality quickly proved that sometimes you're destined to make topaz double-dipped in dung. Dependencies are no exception to this. I've worked with a mapper that works pretty well out of the box. I ran into an issue with how Strings were mapped to boolean values. It turned out the default mapping values for those conversions were yes, no, y, n, and possibly some other variation of those String values. It didn't include true and false by default.

nstart 9y ago
I had a fun one recently. Just the other day in fact. The elastic beanstalk cli from aws was refusing to accept my branches when I tried to refer to a code push from a codecommit repo. Turns out branches with / in it can't be used because it conflicts with the way the cli parses its parameters. Basically it expects a parameter along the lines of codecommit/RepoName/branchName And then it splits the string and if there are more than 3 items as a result it returns from the function saying invalid value. I had to correct it to use it myself but I found it fascinating since it suddenly made the somewhat mystical innards of aws seem a little more human. At least that's how I felt :D.
- pyre 9y ago
  I would submit a bug report. I've reported bugs in aws cli tools before, and they were patched.
  - taormina 9y ago
    They also accept PRs sometimes.

jschwartzi 9y ago

This is especially necessary for driver code from chip manufacturers. Most of that stuff is written to provide demonstrations and test the part, and it should not be trusted.

mixedbit 9y ago

Just a brief study (~few hours) of mysql and postgresql sources allowed me to made a definite decision which of the two is a better choice for all my web projects.

brianwawok 9y ago
Do tell ")
lufte 9y ago
Please elaborate. Did you write a blog post or something like that to document the experience?
agentgt 9y ago
Seriously? I am assuming you are not a database expert but basing this on the look and organization of the code and not the data structures and design. If that is not true than please share your findings instead of teasing us. I'm curious would you do the same with your editor (or IDE)? You should see the source code for those behemoths (with the exception of the rewrites like nvim) they are pretty nasty. Are you going to pick a different editor just because Vim or Emacs (or whatever editor) has nasty code? Guess what IDE has pretty clean code and is extremely modularized... Eclipse. I can tell you Eclipse is not a comfortable editor (and I use it all the time because I'm used to it not because the code is elegant). Mature projects often have pretty ugly code but they are mature and stable. Ugly code also doesn't necessarily mean poor design (e.g. if your editor is single threaded and has nice looking code its still single threaded).
- samastur 9y ago
  There's a difference of category between your and parents example. Editors and IDEs will not make your code crash when it is deployed no matter how flaky or not they are. Databases on the other hand can. I wouldn't imagine it being controversial to prefer well engineered code over less so for bits you will need to depend on.
  - 9y ago

TearsInTheRain 9y ago

You depend on so much code, how is this even remotely feasible?

daliwali 9y ago
This is expected in some fields such as embedded programming, it's reasonable to look at source code just to figure out how hardware works. Or web front-end where you shouldn't be sending huge payloads to the browser anyways. You don't always need to look at the lowest level, just direct dependencies.

lsiebert 9y ago

Crappy documentation for dynamically typed code has often made me read my dependencies. Statically typed languages are much more likely to be self documenting, though occasionally, with Golang for example, they will return a structure with the same name as another structure in the program, and it's hard to figure out which of these is being returned.