4 ms·

Try living off the grid (ala any bank) and see how easy it is for online dependant build systems. Java maven stuff is bad enough already

Freak_NL 10y ago
Maven facilitates downloading all dependencies in one go, exactly for this scenario: mvn dependency:copy-dependencies And when you build a project once, all downloaded dependencies are cached locally indefinitely. That is, you can furnish a Maven build with offline dependencies. Normally you do want to simply have your build download (and cache locally) a new version of a dependency, but an internet connection is not required if you have the dependencies available locally. Maven will always look in your local repository first, and will not try to download anything, except if you are depending on a SNAPSHOT version of something (which implies that the build should always check for a newer version).
- forgottenpass 10y ago
  exactly for this scenario Not really. Building from local sources only or "living off the grid" as parent poster put it typically isn't just about temporary loss of Internet connectivity, but instead serves as a shorthand for a more complicated set of requirements. Generally speaking there will be (at least) disaster recovery requirements such that a computer with a fresh OS install and access to the local very-well-backed-up archives can produce a build. And the local archives typically have exactly one entry point so that legal and process controls can be applied as they go in. So the problem isn't how to cache dependencies for a working directory, but how to get, archive them, and distribute them to a development team easily. A lot of package mangers approach supporting this kind of use case, but still need a fair bit of customization and manual steps if not new tooling built around them. And all that infrastructure also has to meet qualification criteria (through testing). It gets to be enough that any tool that decides to add in their own half-assed download manager will be quite annoying to anyone that doesn't use "YOLO" as their quality management system.
  - berdario 10y ago
    This was the comment that most clearly explains the issue, I'd say. I think that this is an issue of properly covering a use-case, and having simple configuration options to avoid manual and cumbersome steps... But I don't think this means that a build system shouldn't default to downloading its deps. Basically, if "build systems should only build" means - build systems would be better off with an offline build mode available I agree 100%... Otherwise, if it's - build systems shouldn't have an option to automatically download the dependencies and then build I'm still in disagreement
    - forgottenpass 10y ago
      Ignoring what any particular toolset chooses to do, building software and package management can conceptually be separated. Of course they get intermixed in practice, package management provides build prerequisites, and completely separate tools are not necessarily conductive to getting information from the build system (or dependency list in the tarball) to the package manager. Or for the download/installation provided by the package manager to be at the perfect scope for the software build. I, unlike grandparent poster, am not making statements of "ought" for build systems. Just providing context. However, I can still generically say, the tightness of the build and package management limits user flexibility. I'ts not even a particularly interesting phenomenon. It's just another case of software coupling. There are benefits to high coupling and benefits to low coupling. Typically tight coupling means the normal use case works better at the cost of all abnormal use cases. It's the same-old-same-old of all software design. ---- As a side note, I will say that in my experience an inflexible or awkward to use "offline" build mode is not an offline build mode. If "offline build mode" means: -The tarball contains (or build system generates) a list of files that can just as easily be fetched with `wget -i deps.txt` and dropped into a subdirectory manually. That's cool. If it's: -I have to reverse engineer a one-off package manager architecture, write a script to fetch build-deps externally and hack the build to support my particular flavor of offline build because it's not the type of offline build envisioned by the tool author Then that's bunk.
      - lmm 10y ago
tupshin 10y ago
This capability is still a work in progress for rust/cargo, but check out http://doc.crates.io/faq.html#how-can-cargo-work-offline http://doc.crates.io/faq.html#how-can-cargo-work-offline
- tedmielczarek 10y ago
  Note that for the Firefox build we're using cargo-vendor to store all of our crates.io dependencies in the Firefox tree: https://github.com/alexcrichton/cargo-vendor/ https://github.com/alexcrichton/cargo-vendor/ https://dxr.mozilla.org/mozilla-central/source/third_party/rust https://dxr.mozilla.org/mozilla-central/source/third_party/r...

Freak_NL 10y ago

Maven facilitates downloading all dependencies in one go, exactly for this scenario: mvn dependency:copy-dependencies And when you build a project once, all downloaded dependencies are cached locally indefinitely. That is, you can furnish a Maven build with offline dependencies. Normally you do want to simply have your build download (and cache locally) a new version of a dependency, but an internet connection is not required if you have the dependencies available locally. Maven will always look in your local repository first, and will not try to download anything, except if you are depending on a SNAPSHOT version of something (which implies that the build should always check for a newer version).

forgottenpass 10y ago
exactly for this scenario Not really. Building from local sources only or "living off the grid" as parent poster put it typically isn't just about temporary loss of Internet connectivity, but instead serves as a shorthand for a more complicated set of requirements. Generally speaking there will be (at least) disaster recovery requirements such that a computer with a fresh OS install and access to the local very-well-backed-up archives can produce a build. And the local archives typically have exactly one entry point so that legal and process controls can be applied as they go in. So the problem isn't how to cache dependencies for a working directory, but how to get, archive them, and distribute them to a development team easily. A lot of package mangers approach supporting this kind of use case, but still need a fair bit of customization and manual steps if not new tooling built around them. And all that infrastructure also has to meet qualification criteria (through testing). It gets to be enough that any tool that decides to add in their own half-assed download manager will be quite annoying to anyone that doesn't use "YOLO" as their quality management system.
- berdario 10y ago
  This was the comment that most clearly explains the issue, I'd say. I think that this is an issue of properly covering a use-case, and having simple configuration options to avoid manual and cumbersome steps... But I don't think this means that a build system shouldn't default to downloading its deps. Basically, if "build systems should only build" means - build systems would be better off with an offline build mode available I agree 100%... Otherwise, if it's - build systems shouldn't have an option to automatically download the dependencies and then build I'm still in disagreement
  - forgottenpass 10y ago
    Ignoring what any particular toolset chooses to do, building software and package management can conceptually be separated. Of course they get intermixed in practice, package management provides build prerequisites, and completely separate tools are not necessarily conductive to getting information from the build system (or dependency list in the tarball) to the package manager. Or for the download/installation provided by the package manager to be at the perfect scope for the software build. I, unlike grandparent poster, am not making statements of "ought" for build systems. Just providing context. However, I can still generically say, the tightness of the build and package management limits user flexibility. I'ts not even a particularly interesting phenomenon. It's just another case of software coupling. There are benefits to high coupling and benefits to low coupling. Typically tight coupling means the normal use case works better at the cost of all abnormal use cases. It's the same-old-same-old of all software design. ---- As a side note, I will say that in my experience an inflexible or awkward to use "offline" build mode is not an offline build mode. If "offline build mode" means: -The tarball contains (or build system generates) a list of files that can just as easily be fetched with `wget -i deps.txt` and dropped into a subdirectory manually. That's cool. If it's: -I have to reverse engineer a one-off package manager architecture, write a script to fetch build-deps externally and hack the build to support my particular flavor of offline build because it's not the type of offline build envisioned by the tool author Then that's bunk.
    - lmm 10y ago

tupshin 10y ago

This capability is still a work in progress for rust/cargo, but check out http://doc.crates.io/faq.html#how-can-cargo-work-offline http://doc.crates.io/faq.html#how-can-cargo-work-offline

tedmielczarek 10y ago
Note that for the Firefox build we're using cargo-vendor to store all of our crates.io dependencies in the Firefox tree: https://github.com/alexcrichton/cargo-vendor/ https://github.com/alexcrichton/cargo-vendor/ https://dxr.mozilla.org/mozilla-central/source/third_party/rust https://dxr.mozilla.org/mozilla-central/source/third_party/r...