6 ms·
A lot of apps (old-timey Windows apps, for example) have this philosophy, leading them to reinvent things like crypto and image decoding. Naturally, this leads
by pcwalton 11y ago
A lot of apps (old-timey Windows apps, for example) have this philosophy, leading them to reinvent things like crypto and image decoding. Naturally, this leads to tons of bugs, including security bugs.
I would revise this to: Don't bring in more code than you need. But if the choice is between writing something yourself and using someone else's well-tested, heavily-used library, always go for the latter.
- ktRolster 11y agoAs an architect, you need to be able to do a cost/benefit analysis of each option. That is what software architects do, why they have experience. For example: How much time will it take to implement each option? How much time will it take in the future to support it? What security risk does each option incur? What is the risk of the project being abandoned? What is the risk of the project changing in non-backwards compatible ways? What are the performance characteristics of each option? NIH is a disease, but so is import-mania. With experience, you can make a good decision.
- mathattack 11y agoGood engineering managers and architects are great at balancing both biases.
- maxxxxx 11y agoBalance is good. Unfortunately ideologues have taken over software as they have taken over politics.
- ben0x539 11y agoAre there really people out there whose decisionmaking process goes like "well we don't really need this library, but I'm gonna depend on it anyway to further the ideology of our movement"
- maxxxxx 11y agoI know plenty of people who want to port everything to JavaScript. When you ask "Why?" they don't know. That's an ideology.
- the_af 11y agoIn what sense are technical preferences you cannot explain or justify an "ideology"? Ideology is not unjustified/mistaken belief. Maybe you mean cargo culting in technology, of which there is sadly a lot? Or maybe even "let's do this in JavaScript, because it's the only tech I know and I can't be bothered to learn something new"? That happens, but it's not ideological.
- ktRolster 11y agoYou're defining ideology too narrowly.....an ideology is a system of ideas and ideals. It doesn't have to be mistaken, and it can be completely justified and correct. "I think Javascript is better because I prefer to use technologies I know" can be part of an ideology.
- the_af 11y agoIt was the OP, not me, that implied ideology is something negative and linked to poor justifications. For the record, I think ideology is (or can be) a good thing in politics, but a misappropriated term in technology. > "I think Javascript is better because I prefer to use technologies I know" can be part of an ideology. But that's not what the OP said. Instead, it was "I think we should use Javascript because... uh... I dunno. Let's just do it!". That's not ideology by any possible meaning of the word.
- mathattack 11y agoIdeologies are rarely presented as such. I have seen places where they wrote databases from scratch, and rewrote Windows scrolling bars. Always 100 good reasons why using something else wasn't good enough. Other times I've seen people always want to buy the shiny nice toy rather than write a few lines of code themselves.
- krinchan 11y agoAlso de-duplicating dependencies is pretty big. For instance, in Java land, I have a project built in Camel, and Camel is deeply in love with the Jackson JSON parser. So I use it there. On the other hand, I've been learning to use GSON's parser in my Sponge plugin (a Minecraft server) because the SpongAPI dependency pulls that in anyway. Albeit, both libraries are dead simple to use so it's a bit contrived, but I see a lot of projects that would pull in Spring's RESTTemplate into a Camel project when they've already pulled in CXF or have Apache's HTTPClient readily available via other dependencies. (And no, URLConnection is terrible. TERRIBLE.)
- skewart 11y agoOne thing I've gotten into the habit of doing is looking around the commit history and issue list for any package I import. Was it something somebody wrote in a hurry and hasn't really touched since? Is it something that has a solid set of regular contributors? Are there a lot of outstanding issues relative to how heavily used it is? I also spend more time actually reading through specs to see how well they exercise the code. That's probably standard procedure for a lot of people, but it's something that I had to learn to always do.
- ktRolster 11y agoThat's a good idea
- edgyswingset 11y ago> Don't bring in more code than you need. I see it as a sliding scale. If I'm parsing 1 string with the same date format into 1 object, I'm not going to pull in some general purpose time parsing library - I'll write the 10 lines of code myself, a few unit tests, and be happy. If in the future I start having to deal with different date strings and some need to do more than just throw up a single date on a page somewhere, I'll get a date/time library.
- amelius 11y agoYou could also use the third party library for your unit tests.
- deleted 11y ago[deleted]
- jvehent 11y agoThis is pretty much what Rob Pike advocates in Go: "A little copying is better than a little dependency." http://go-proverbs.github.io/ http://go-proverbs.github.io/
- pcwalton 11y agoI pretty squarely disagree with Rob Pike on that one. Copying is how you introduce bugs and insulate yourself from upstream bugfixes. I'm suggesting that you should try to remove code first, add a dependency on well-trusted code if that doesn't work, and only copy/reinvent as a last resort.
- ktRolster 11y ago> I pretty squarely disagree with Rob Pike on that one. TBH that's kind of an indication that you should rethink your position. Rob Pike has a lot of experience, he's seen a lot, and he knows what he's doing. I'm not saying you're wrong, just that you shouldn't snap to the defensive position.
- pcwalton 11y agoRob Pike's authority does not counter the negative experience I've had with vendoring dependencies, making local changes, and drifting so far from upstream that it becomes extremely difficult to merge in bugfixes. Or the positive experience I've had with package managers making it easy to bring in third-party code, easier than copying it in.
- maxxxxx 11y agoThe keyword is "well-tested, heavily used library'. Especially in the web area I see a lot of imported libraries from github or wherever for doing simple things. In the end you end up with dozens of dependencies you don't know how and when to update. My rule of thumb is if the stuff we need can be reduced to a few functions, it's better to copy so you at least know which code you are using and don't have thousands of lines of code in you repo where you don't know if they are ever being used,
- pcwalton 11y agoCopying doesn't make anything better; it just insulates you from upstream bug fixes. If copying is better than adding a line to your Gemfile or whatever, then that's a usability bug in your package manager. The entire reason for package managers' existence is to provide an easier-to-use, more reliable alternative to copying.
- nickpsecurity 11y agoThought they also eased deployment and updates.
- Retric 11y agoUpstream updates can add bugs just as easily as bug fixes.
- acdha 11y agoThat's an argument for having an effective review and testing process. Change is inevitable and it's better to be good at doing it routinely than putting it off until an emergency.
- Retric 11y agoThis is overhead for every update of every library. In theory it's a great idea and expensive idea, so of course nobody does it. There are two mindsets in coding, this code needs to work right now and this code needs to work in 20 years. Linking code is very likely to break in the second time frame. Public API's are generally unstable, services goes away, and people break things. But, if all you need is a toy demo then feel free.
- mbrock 11y agoThere's an important kind of compromise that isn't discussed as often: using an external library but behind an interface of your own design, an "anti corruption layer". If the external dependency is limited to one small bridge in your application, then it's so much easier to see what parts of the dependency you actually depend on, to upgrade the dependency when its API inevitably changes, to replace it entirely if it becomes a burden.
- rch 11y agoI'll link to the facade pattern, since last time I mentioned it people went a week or so before realizing it wasn't a term of my own invention: https://en.wikipedia.org/wiki/Facade_pattern https://en.wikipedia.org/wiki/Facade_pattern
- jermo 11y agoThe downside is that you bring in another layer of indirection and, as a result, greater cognitive load for yourself. Mycode -> facade -> library Navigating code becomes more cumbersome and stack traces longer. I do like this approach too but it isn't free.
- brianwawok 11y agoHah I stopped worrying about stack trace length when I started writing Scala ;) Usually only 1 or 2 lines of the trace matter, you learn to skip the rest pretty fast.
- chris_wot 11y agoHere's an interesting story from the Java world - "Filtering the Stack Trace From Hell": https://dzone.com/articles/spring-vs-java-ee-the-real-story-comic https://dzone.com/articles/spring-vs-java-ee-the-real-story-... An even more breathtaking stack trace image can be found here: https://ptrthomas.files.wordpress.com/2006/06/jtrac-callstack1.png?w=630 https://ptrthomas.files.wordpress.com/2006/06/jtrac-callstac... And the PDF version: https://ptrthomas.files.wordpress.com/2006/06/jtrac-callstack.pdf https://ptrthomas.files.wordpress.com/2006/06/jtrac-callstac...
- pdkl95 11y agoWhile I completely agree with the sentiment[1], there is a bit of hyperbole (and/or literary license) in the suggestion to "Kill Your Dependencies". Modular, well-contained code is very good. It's usually a good idea to build on other people's work, though this is yet another trade-off decision that will always be part of the software design process. A lot of the dependencies discussed in this blog post are libraries that aren't actually adding anything useful: the various JSON parsing libraries that should be replaced with the parser in stdlib, or rspec testing libraries that shouldn't have ever been a regular runtime dependency. [1] Managing complexity and dependencies is probably the most important concern going into the future, not just in programming, but also in every other complex system.
- skewart 11y ago"But if the choice is between writing something yourself and using someone else's well-tested, heavily-used library, always go for the latter." Absolutely. However, there are plenty of situations where what you pull down from npm or rubygems isn't actually all that well-written or well-tested. When I first started programming I kind of had this impression that if an open source library is published on a package repo and people are using it then it must be much better than anything I could write. I have learned the hard way over the years that is not always true.
- jpulec 11y agoI feel like I relearn this lesson at least once a month.
- moron4hire 11y agoSo many libraries in the wild aren't "well-tested, heavily used", though, and sometimes it is really hard to differentiate between popular and good. At the end of the day, the only person who is responsible for the quality of your proje t is you. You have to figure out which parts of your project are key to your operation and which are just window dressing. If your job is to make a blogging platform, I would expect many features of formatting documents to be reimplementations of other people's work, because you have to know you are relying on yourself for your core purpose. I also don't understand the heartburn other developers have over knowing "OMG L, THERE IS SOMEONE ON TEH INTARWEBS AND THEY ARE REINVENTING THE WHEEL". It smacks of fear, a fear that is ultimately rooted in insecurity. If a person was secure in their knowledge of their skills, their ability to understand problems and fix them, then there should be nothing to fear from a dozen or a million different libraries doing the same thing and running into one or two of them on one's next project. It is just a matter of course.
- pcwalton 11y ago> If a person was secure in their knowledge of their skills, their ability to understand problems and fix them, then there should be nothing to fear from a dozen or a million different libraries doing the same thing and running into one or two of them on one's next project. I actually do fear a million reimplementations of, say, RSA.
- moron4hire 11y agoThe impact of bad software is vastly overrated. Almost all of it is bad already, and yet we haven't killed ourselves off as a species yet.