9 ms·

Show HN: Duo – a next-generation package manager for the front end

• compedit 12y ago
  You can require dependencies and assets from the file system or straight from GitHub: Oh my. You've essentially killed Bower and NPM in one swoop then haven't you? An exciting time to be a web developer to say the least, this looks simply amazing.
  • bigethan 12y ago
    well, except for the `npm install -g duo` bit :-)
  • qwer 12y ago
    npm already does this from any git repo. I have no idea why anyone would want to tie themselves to github only.
• neilellis 12y ago
  I love simple things, this gets my +1, look forward to learning more. I can't see why this is browser only, am I missing something?
  • michaelmior 12y ago
    I'd imagine doing this for node.js would require modifications to node itself since the behaviour of require would need to change. On second thought, I suppose you could just rename packages and mess with their path in node_modules to make things work. But that would mean moving out of the npm ecosystem which seems rather bold.
    • nawitus 12y ago
      Duo.js could implement duo_require for Node.js. Every "Duo.js Node.js" app would need to be bootstrapped with Node.js require, though.
      • neilellis 12y ago
        It would be beautiful to have such a simple and open package manager for node. Git FTW :-)
• bpatrianakos 12y ago
  I think I'm missing something. I see how this would be great when it's build time but during development I don't want to have to keep running build commands each time I want to use a new package I'm development. Using Grunt and Bower may be a little time consuming up front but once things are set up its very easy to keep a separate Dev and prod environment in sync between different contributors. I only see the value at build time with Duo which is why I'm sure I missed something.
  • michaelmior 12y ago
    Don't you already have to run some command to install the new package anyway? Not really sure I see how this is different in that respect.
  • TheHydroImpulse 12y ago
    You inherently need a build step when working with remote dependencies or even any dependency. Especially when you're working with various assets like templates, css, images, etc... Typically, let's say for an app, you'll run a watcher which will trigger the builder, so you can just spin it up and leave it alone. For smaller components that are meant for consumption, a simple Makefile is more than enough and running the command isn't too terribly bad.
  • tjsix 12y ago
    It looks like the cli utility has a built in watch flag that can be passed to automatically rebuild on changes
  • Kiro 12y ago
    You need to run a build script for Browserify as well, which IMO everyone who is using Bower/RequireJS should switch to.
  • davej 12y ago
    I imagine it would possible to wire up a simple watcher in gulp/grunt that would do this automatically.
• vkal 12y ago
  So earlier this summer I learned how to use RequireJS + a smidgen of Grunt, then I felt the need to move towards Gulp and Browserify (which I've just recently started), and now I'm excited about Duo. It would be interesting for someone with more expertise to do a compare/contrast further down the line of all three. "...I show you how deep the rabbit hole goes" - Morpheus Also, does someone know how the Closure compiler fits into all of this? It might be totally unrelated but I'm trying to learn more about JavaScript application architecture, and I'm not sure where that fits in.
  • munificent 12y ago
    > I felt the need to move towards Gulp and Browserify Dude, you're so behind the times. It's Yeoman and Bower now. Wait, Broccoli and Duo.
    • Gigablah 12y ago
      Yeoman isn't a build tool though, it just generates the application skeleton.
  • cel1ne 12y ago
    I tried a lot and use RequireJS + Gulp now (same Gulpfile on every project). Works for me, I don't see a compelling reason to change something. My Gulpfile has 200 lines and does everything from asset-manifests to css minification, js-reloading etc.
• michaelmior 12y ago
  Personally I think I'd much rather have my dependencies be explicit rather than inferred from what I require across my code base. The idea of a tool which supports both Bower and Component packages is quite nice though.
  • tjholowaychuk 12y ago
    I used to think this too, but the more I use Go the more I enjoy not needing to maintain a silly manifest
    • TheHydroImpulse 12y ago
      Then when you need to pin down versions and such, it's a massive pain (in terms of not being able to have a manifest file in Go's native case).
      • chrisrogers 12y ago
        I think the classic solution of vendoring your dependencies would be the best strategy here.
      • tonyhb 12y ago
        Pote's GVP/GPM works really well here. It's such a small system to learn and use, but does everything you could ever need when versioning godeps: https://github.com/pote/gpm https://github.com/pote/gpm && https://github.com/pote/gvp https://github.com/pote/gvp
    • michaelmior 12y ago
      I keep meaning to spend some time playing with Go, but I haven't yet so I don't know how that feels. However, I wonder what it says that several dependency management tools have emerged for Go. https://github.com/mattn/gom https://github.com/mattn/gom https://github.com/nitrous-io/goop https://github.com/nitrous-io/goop https://github.com/tools/godep https://github.com/tools/godep
      • 12y ago
• yid 12y ago
  Reading the copy on the page, I wonder if the author has heard of npm init and npm install --save. I haven't had to edit package.json in quite a while.
  • sehr 12y ago
    Reading the copy on the page, I get the impression that the author dislikes the idea of a manifest (at all) rather than just the act of editing it.
• colinramsay 12y ago
  This looks superb. Great work.
• zackbloom 12y ago
  I'm not sure this actually improves the flow for me. I (like most sane devs), like to lock my deps to specific versions (or vendor them). Currently, it's just a matter of running: npm install --save dep With this, I either have to be satisfied with not locking down the version, or go lookup the current version manually before adding the reference to my code. It also looks like upgrading a dep would mean changing every require.
• teabee89 12y ago
  The Go link should point to http://golang.org http://golang.org
• Kiro 12y ago
  Things are moving fast in the front-end world. Just as we've stopped using RequireJS and accept Browserify as the new king DuoJS comes and overthrows everything again.
• fiatjaf 12y ago
  Does a simple `require('package')` without the slashes fetch the thing from NPM? Maybe it could. Maybe it could also work with `require('package@0.5.1')` and other sugars like this.
  • matthewmueller 12y ago
    require('package') will look into the manifest to see if there's a remote path and version to use, otherwise it will be set as unresolved and ignored in the build.
• swah 12y ago
  But, but.. I _just_ decided to use webpack :( More seriously: do you feel like it incorporates all the lessons of its predecessors?
• tjsix 12y ago
  Maybe it's just me but having no manifest makes using this tool quite painful. It has taken me 10 mins just to figure out how to structure the require()'s so that it doesn't error out, and that's with only requiring two packages. For example I created a quick index.js file and required angular and restangular. It immediately errored out because the angular.js repo doesn't use semver on it's master branch. Ok fine, switch to angular/bower-angular, nope, it looks for index.js so my require now has to read: require('angular/bower-angular:angular.js'). Run again, error, same issue with restangular, it's looking for index.js. That require now reads: require('mgonto/restangular:dist/restangular.js'). I had to actually find where the files are that I wanted to require and explicitly state them in the require. Shouldn't this automatically parse bower/component/package.json files for this info, especially if you're touting the 'No Manifest' thing.
  • anthonyshort 12y ago
    index.js as the entry point for a module is pretty standard with projects built around Component and npm. It's bower and the lack of the standard there that creates the issue.
    • bakareika 12y ago
      As an author of a (small) number of npm+bower packages I must admit that nope, that's not "standard" in any meaningful way.
      • TheHydroImpulse 12y ago
        It's definitely standard in Node land and also Component(1). Bower, on the other hand, does not and thus is the culprit.
        • skeoh 12y ago
          index.js is only 'standard' in node insofar as it is the fallback entry point if a package.json file is not found or does not specify a 'main' file.
• nawitus 12y ago
  Is there support for symlinking for easy development? I mean the equivalent for 'npm link' or 'bower link'. Also, can you host your own "duo registry" (if the concept makes sense here). And if so, can you point to the private registry instead of using the public GitHub? Bower and npm supports these, and that's what I'm using currently.
  • anthonyshort 12y ago
    For private packages, you'd just make the repo private on Github and make sure you have auth details in your .netrc file. So you don't really need your own "registry". There is talk about supporting more than just Github too.
    • nawitus 12y ago
      That's a rather centralized solution. There are many companies who host their own npm/bower registries, and they won't move to Duo.js until that's possible.
      • anthonyshort 12y ago
        But if you're at that point you probably have your own private git hosting somewhere too (or just private repos somewhere else). When there is support added for different remotes you could just require from there instead of Github. Point is, just use the git repos instead of needing that registry at all.
        • nawitus 12y ago
          Yeah, that's working solution, when support for that is added. Assuming versioning works with these git urls.
    • deleted 12y ago
      [deleted]
• skrebbel 12y ago
  I like the ideas, but it feels like it's just a small incremental step over established solutions like webpack and browserify. Does this have any more to offer other than deducting what to download&install from the dependency tree? Because if that's "all", it would be easily added to e.g. webpack, I suppose. Is it really worth making yet another dependency and build tool just for that one feature? If you build on top of webpack, you get a lot of stuff for free, like speedy and dependable file watching, hot-reloading development servers, support for nearly any imagineable frontend language, and a remarkably decent extensible architecture. All this has to be made again for Duo. Both dependency management and frontend building are highly complex tasks. I'm not saying that therefore it couldn't be done better, but I do honestly, without judging, wonder whether the authors seriously considered existing solutions and ran into impossible problems, or whether this is just the Not Invented Here syndrome at work.
• wycats 12y ago
  It sounds like this means that if I wanted to use a specific version of Ember, I would have something like the following line in many files: require("emberjs/ember-component@^1.6.1") Doesn't this mean that to update an Ember package, I have to change every file that references that package? That would mean having to touch the JS file for every component in your project to bump the version. Am I missing something?
  • anthonyshort 12y ago
    You'd use a component.json to lock it down for an app. The inline versioning is really just a way to quickly write code and get something up and running. Manifests are just a pain when you're just getting started with a project or a small script. If a component.json is present, it'll use that to get the versions and you just require('emberjs/ember-component')
    • wycats 12y ago
      Ah. I would probably clarify the language to make it clear that having a manifest approach is the intended approach for real-world apps, but that it can be left out when playing around.
  • matthewmueller 12y ago
    The manifest is optional, so for big projects I would probably recommend using a manifest to keep all your versioning in one place. Duo falls back to the manifest if it cannot resolve the path on it's own. For proof of concepts and hacks though, adding a manifest is a waste of everyone's time.
    • wycats 12y ago
      I personally haven't had much trouble with: npm install foo --save-dev When you go beyond a single developer playing around, having a quick, well-known place to look to learn what dependencies are being used is very useful.
  • TheHydroImpulse 12y ago
• RoboTeddy 12y ago
  Is it fast during development on large codebases? (e.g. by compiling incrementally)
  • TheHydroImpulse 12y ago
    The slow part is fetching dependencies. The "compiling" isn't compiling in the traditional compiled-language sense, where that would most likely take a significant amount of time.
    • RoboTeddy 12y ago
      The tool might have to recursively parse each file into an AST to find the `require` statements, and potentially apply various transforms (e.g. coffeescript). On large frontend codebases, these tasks can take an annoyingly long time. Hence, some tools cache (e.g. see browserify vs. watchify).
      • ianstormtaylor 12y ago
        Yup, it will cache the files automatically and check mtime's so that only the changed files need to be reparsed for dependencies. It ends up being really fast to build once dependencies are installed.
        • RoboTeddy 12y ago
          Fantastic!
• tomdale 12y ago
  One question I always ask when looking at a package manager is, "Will this help me have reproducible builds?" (http://martinfowler.com/bliki/ReproducibleBuild.html http://martinfowler.com/bliki/ReproducibleBuild.html) I've worked on enough large projects that relied on prayer and optimism instead determinism when resolving dependencies, and eventually you end up losing hours or days trying to get the app running on a new server or a new developer's box. Not requiring a manifest is a bug, not a feature. If people can specify dependencies, and information about the version used is lost, then you can be sure that it will happen in practice. Code designed for "proofs of concept" has a funny way of making it into production. As we know from studies of things like organ donation rates, even the smartest humans get tired or distracted and make bad decisions. The only guard we have against it is choosing sane defaults. (See Yehuda Katz' RailsConf keynote for some interesting insight into how this interacts with convention over configuration: http://www.confreaks.com/videos/3337-railsconf-keynote-10-years http://www.confreaks.com/videos/3337-railsconf-keynote-10-ye...). While the simplicity of DuoJS allows it to win the Pepsi Challenge against other options, where the first sip tastes very sweet, I would never again willingly choose a dependency manager that makes reproducibility anything other than mandatory.
  • TheHydroImpulse 12y ago
    https://github.com/duojs/duo/issues/220 https://github.com/duojs/duo/issues/220
  • agumonkey 12y ago
    Talking about reproducible builds : is there something better than nix http://nixos.org/nix/ http://nixos.org/nix/ ? (honest question)
    • CMCDragonkai 12y ago
      Nope. Nix is the best.
  • matthewmueller 12y ago
    Author here. @tomdale We're on the same page. I've worked with many large JS codebases as well and consistency and repeatability are a must. However, as with any tool, it's effectiveness comes from how you use it. There are so many NPM modules that use "*" in their manifests. That's not very robust. Having a manifest isn't a solution, it's how you use the manifest. In Duo you can pin down any dependency right in the file via require('some/repo@tag'), which is the same as having a manifest with a pinned version. With that being said, I'm open to any and all contributions or ideas on how to make Duo more consistent and repeatable. I love what Docker does for it's image builds and I think if we could get that kind of robustness going on the frontend, we would all benefit.
• danielnaab 12y ago
  npm supports git repositories as dependencies already (git://github.com/...), so that feature doesn't seem like much added value to me. And considering that any real project needs a manifest, the versioning syntax doesn't seem especially compelling. I'd like to be proven wrong, because this looks cool... but I don't see the value over npm and browserify other than some syntactical sugar (which could probably be achieved with an npm wrapper). Am I missing something?
  • matthewmueller 12y ago
    don't remember the details, for simple repos it's fine, i think it's a dependency of dependency problem, that isaac's didn't want to support for some reason. here's a rabbit hole of additional information: https://github.com/npm/npm/issues/3014 https://github.com/npm/npm/issues/3014 :-)
• aymericbeaumet 12y ago
  Is Duo smart enough to remember when it finds a library requirement with a specific tag? So that it is not necessary to indicate the tag on the other places where this same library is required. If so, manifests could be written in a very neat way. See: https://gist.github.com/aymericbeaumet/22c3a9deba54549821e3 https://gist.github.com/aymericbeaumet/22c3a9deba54549821e3
  • anthonyshort 12y ago
    You could also just export from each of the requires in that manifest.js file you have there. Although there's no real benefit of doing that over using the manifest other than it's kinda cool :)
    • aymericbeaumet 12y ago
      A one could indeed export its requires. However, my idea was was to stay close to the require caching system existing in Node. As a personal taste, I by far prefer to configure my application in a JavaScript file rather than a JSON file. And I consider dependencies as a part of the configuration process. It agrees with the Duo philosophy which aims to simplify the building process from A to Z.
  • ianstormtaylor 12y ago
    That is a pretty cool idea! I wouldn't have even thought to do that
• progx 12y ago
  Did Duo the same as gulp+browserify ?
• andreaf 12y ago
  I am a little confused about how Duo actually transforms dependencies into JS values. When I write, say, var uid = require('matthewmueller/uid'); as in the home page example, what gets bound to the uid identifier? The point is that a package manager does not only need to fetch dependencies, but also to specify relations between modules. This is why, for instance, Bower only does half the story (fetching) and it has to be coupled with a tool like Require.js to actually provide modules.
  • andreaf 12y ago
    For clarity, let me make a comparison with the JVM. On the JVM, when one requires a class in a package, it is up to a classloader to find it. The default classloader will look up the class code in a directory structure based on the package name. Where to locate this directory structure is decided based on environment variables, but usually it ends up to having multiple entry points, either on the filesystem or inside zipped files called JARs. A separate tool, like Maven on SBT, can help you fetch JARs from repositories. But what actually makes everything click is the fact that once you have fecthed the JAR, you know where to locate classes inside it based on their name. In the frontend world there just isn't such a standardization on paths, and this is what makes it difficult to locate a module even when the Github repo for it has been downloaded. How does Duo solve this issue?
  • matthewmueller 12y ago
    The flow is... 1) Download https://github.com/matthewmueller/uid https://github.com/matthewmueller/uid from github 2) Find it's entry using https://github.com/duojs/main https://github.com/duojs/main 3) Parse it's entry, and extract the dependencies 4) Recurse
• deleted 12y ago
  [deleted]
• theefer 12y ago
  People interested in Duo may also want to have a look at jspm.io. It solves a similar problem, but with a few differences which to me are advantages: - Transparently supports modules from CommonJS, AMD, ES6 or globals. - Enforce a manifest (config.js) that let you pin dependencies (incl. transitive dependencies) to exact versions. Unlike RequireJS config, jspm automatically manages that file for you. - Support multiple package providers, e.g. NPM, on top of Github. - Based on SystemJS, a polyfill for the upcoming standard System loader. This hopefully makes it future-proof. - Does not require a compilation step: dependencies can be pulled dynamically from a CDN over SPDY. Alternatively they can be cached locally as well. A compilation step (jspm bundle) is still available. - Works both in the context of Node and the browser. We've been successfully using jspm and SystemJS in production at the Guardian. It's still early days, but the devs are very active and responsive. This isn't meant to distract people away from taking a look at Duo and making up their own mind, but I noticed nobody mentioned jspm in this thread and thought people may want to look at both and compare.
• andrezsanchez 12y ago
  I have a hard time seeing what exactly the point of having another package manager is. We already have a very established way of managing front-end packages: NPM and Browserify. I don't see why you would want any of the features that are advertised in this package manager, namely using Github repos vs. npm packages, and manifestless packages. All it gives you is a more fractured ecosystem and code that won't run on the server.
• stephenway 12y ago
  Just released a new library so Sass users can now use Duo.js with duo-sass (https://github.com/stephenway/duo-sass https://github.com/stephenway/duo-sass). Check it out!