12 ms·
Show HN: WAPM – Package Manager for WebAssembly
- moosingin3space 8y agoI love WebAssembly/WASI, but I'm concerned about us repeating the mistakes of the past. How does the dependency model work? Can I safely install parallel streams of software? Does it support unprivileged installation? Is the installation stateless (no scriptlets/lifecycle scripts)?
- MuffinFlavored 8y ago> Is the installation stateless (no scriptlets/lifecycle scripts)? I feel like the reason `npm`/the node.js ecosystem has these is because... as soon as you want to do anything serious, you need them. How else do you build native modules across different platforms, run initial post-installation configuration scripts, etc.?
- numbsafari 8y agoYou make it declarative instead of scripted. Scripts aren’t truly platform independent. You’ll branching logic for different platforms. Why not capture that with different declarative structures for each platform? Running scripts as part of software install is something we, as an industry, need to solve.
- moosingin3space 8y ago> How else do you build native modules It is my understanding that wasmer is trying to create a new ecosystem that doesn't rely on native modules. > run initial post-installation configuration scripts It's been my experience, as a long-time Linux user, that this is actually a bad thing. Stateless systems are far easier to work with -- a package manager can be far faster and simpler if it just extracts an archive. I can't think of a single case where post-installation configuration scripts couldn't be replaced by something simpler, using the filesystem.
- orf 8y agoDownloading a dependent native library from an official source, based on the current OS version.
- moosingin3space 8y agoCould this not be done though the dependency system itself, if it were sufficiently generic?
- xmclark 8y ago> How does the dependency model work? Currently dependencies are keyed by namespace, name and version. There is a global namespace that has restricted access. WAPM does not do any dynamic linking of WebAssembly and currently only resolves dependencies one node deep. This will likely change in the future as the story on WebAssembly libraries and dynamic linking becomes more concrete. > Can I safely install parallel streams of software? WAPM operates synchronously at the moment, but there is no reason why WAPM should not be able to install dependencies concurrently. > Does it support unprivileged installation? WAPM installs packages into a "wapm_packages" directory in the current directory. WAPM will probably support global installs in the future. WASI enables a "capability-oriented system", but this is a concern of wasm runtimes, and not WAPM. WAPM only manages wasm binaries. > Is the installation stateless (no scriptlets/lifecycle scripts)? WAPM installs with a single command and there are no lifecycle scripts. Ideally, one would install their wasm packages with wapm-cli and not require any intervention from other tools. It was designed to be unobtrusive.
- moosingin3space 8y ago> WAPM operates synchronously at the moment, but there is no reason why WAPM should not be able to install dependencies concurrently. I'm sorry, I didn't word that well. In Fedora, there is a feature called Modularity that enables you to switch between different release streams (for instance, Node.js LTS or current). You can do so with `dnf module install nodejs:11` or `dnf module install nodejs:10`. Since `dnf` is installing into a single global space, though, it isn't possible to install both nodejs:11 and nodejs:10, and the Fedora project recommends containers as a solution here. Could WAPM support this use case, without requiring one to buy into the entire container tooling ecosystem? > no lifecycle scripts Yes! This could enable a declarative OCI image build tool, which could be used to bridge the gap between Wasmer and the Docker/Kubernetes stack.
- syrusakbary 8y agoHi HN! We have also published an article to showcase wapm: https://medium.com/wasmer/announcing-wapm-the-webassembly-package-manager-18d52fae0eea https://medium.com/wasmer/announcing-wapm-the-webassembly-pa... Would love to hear your thoughts!
- phillmv 8y agoHello, 1. Package managers are very hard! The same problems wrt dependency resolution pop up again and again. Whatever you do, I recommend uhm, copying whatever Yehuda et al did for Ruby, Rust, Yarn, i.e. bundler/cargo/yarn There's lots of prior art out there, and a few days of reading can save you months of terrible effort! Here's one example but there's loads more out there https://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn-the-new-js-package-manager-2/ https://yehudakatz.com/2016/10/11/im-excited-to-work-on-yarn... 2. Namely, I don't see any lock files! I didn't look very hard. But get yourself some lockfiles. I also can't stress this enough. 3. Packaging signing really needs to be baked in from the get go, re: https://news.ycombinator.com/item?id=19733085 https://news.ycombinator.com/item?id=19733085). Good luck!
- whyrusleeping 8y agoAre all the 'packages' here reproducible? I'd love to be able to run something like `wapm verify <package>`, or `wapm verify --all`. Running arbitrary code from random people on my computer is sketchy even when I can look at the code. Its far more sketchy when I can't.
- syrusakbary 8y agoWe are working on supporting signed packages to assure they can't be tampered with. At the same time WebAssembly provides some nice sandboxing capabilities and we are working to add permissions on top of syscalls, so packages will not do what they are not suppose to do. Reproducible builds are definitely something great, but they are quite tricky. However we are very open to hear more thoughts on how to do it!
- whyrusleeping 8y agoSigned packages are useful, but still don't solve the problem. One fairly simple thing you can do to improve this would be to include build scripts as part of the package, and allow people to run those build scripts through wapm. The exact versions of any involved tools will need to be recorded too, but since there arent too many ways to generate wasm blobs yet, this shouldnt be too out there. Eventually it would be great if everyone had fully reproducible build formulae (kinda like https://github.com/polydawn/repeatr https://github.com/polydawn/repeatr).
- est31 8y agoYeah, instructions onto how to build the package would be highly valuable. Environments where binaries are very backwards compatible (which itself is a good thing!) lend themselves to discarding abilities about how to build something. You could quickly land in a situation where you need to recompile some particular binary for some reason but figuring out how to compile it is really hard. Like how Microsoft fixed a vulnerability in an Office program by changing the binary instead of the source code. IMO any open source centric repository should have developers upload the source code instead of binaries so that the repository can compile it themselves to give binaries or source code to users. At the start they could use docker where the docker file is part of the uploaded artifacts, and later they could use WASI based toolchains directly, e.g. clang compiled to wasm or rustc compiled to wasm. This "developers upload binary artifacts, source code is an afterthought" idea of npm rubs me wrongly.
- JeremyBanks 8y agoI like Wasmer, but I very much dislike the idea of another major package repository being owned and operated by a commercial entity. They seem like good people doing valuable work now, but that doesn't tell us very much about who the company will be in five or ten years. I've grown uncomfortable with NPM being operated by NPM Inc instead of The Node.js Foundation, but it's a hard thing to change once it's established. We should hesitate to support the establishment of another community package manager by a for-profit company.
- syrusakbary 8y agoI completely understand your concerns, especially given the issues that NPM had lately. We are an open-source focused company and our success is tightly tied to the success of our open-source solutions. Here are some of the things that we are doing in order to resolve (or minimize) the concerns: * Completely open API to retrieve the registry data (it's a work in progress, but our GraphQL API is open to everyone to use: https://registry.wapm.io/graphql https://registry.wapm.io/graphql ) * De-centralized hosting of the packages (news to come soon!)
- yuvipanda 8y agoThanks! This is a good start. Do you have ideas on how you will make the governance of the registry open? Who will make important decisions around policies, and how will they be made? I personally find this to be the core reason I find it hard to use startup-run package registries.
- deleted 8y ago[deleted]
- syrusakbary 8y agoWe are starting to research on governance, but to date nothing is defined (since we have just launched). However, I would love to follow up with you personally to make sure we make the right choices from the start. Can you write me an email to syrus@wasmer.io so we continue the conversation there?
- localhostdotdev 8y agoanyway, sounds really cool and hoping it gets inspired by things like bundler/homebrew more than npm/yarn. btw it has "telemetry" enabled by default. edit: thanks for the link, didn't find the source previously
- syrusakbary 8y agoThe CLI is open-source: https://github.com/wasmerio/wapm-cli https://github.com/wasmerio/wapm-cli We are also planning to open-source the registry as well :)
- deleted 8y ago[deleted]
- justinclift 8y agoAhhh cool. The GitHub link on the bottom of the page just goes to this empty GitHub repo: https://github.com/wapmio https://github.com/wapmio So I initially thought it was all closed source. Maybe add more GitHub repo links, so people can find the right stuff? :)
- catern 8y agoWhy a new package manager? Why not port an existing well-designed generic package manager such as Nix? Making a new package manager is very expensive in terms of developer times - developers have to learn yet another new thing to use your platform. Can you justify this? Edit: For example, as matthewbauer commented below, Nix can support WebAssembly as a compilation target: https://github.com/NixOS/nixpkgs/pull/56555 https://github.com/NixOS/nixpkgs/pull/56555
- Godel_unicode 8y agoWhich general purpose package manager works correctly on Windows (not WSL, actual Windows) and Linux? Cross-platform is more important than not creating another package manager. I think a far more interesting question is, does the package repository support falling back to e.g. curl. That would allow those who don't want another package manager to still use the registry.
- chuckdries 8y agoThey say the registry info is available via API, so perhaps a simple bash script that calls curl would suffice
- dman 8y agoconda
- Godel_unicode 8y agoI always forget about conda
- rkeene2 8y agoAppFS [0] is cross-platform and general purpose. It should work on Windows via cxfuse [1], but I have not tested it. The data structure is simple and could also be handled by an offline fetching system. [0] http://appfs.rkeene.org/ http://appfs.rkeene.org/ [1] https://github.com/crossmeta/cxfuse https://github.com/crossmeta/cxfuse
- bayesian_horse 8y agoThe documentation such as it is, doesn't even tell me what this is supposed to do (or at least not after a superficial look). Is it for using web assembly binaries on a command line? Or in the browser? Integration with webpack or the like? There is a link on how to install from source, pointing to github, but the link is dead and the organization has no repositories... At the moment, there is not enough momentum behind individual web assembly packages, as in "many people wanting to include the same wasm binaries". If at some point there are runtimes for like C#, Java, Python etc which people agree on, they might try and use a CDN approach, such that users visiting multiple websites don't need to download everything at every site.
- sitapati 8y agoIt's a Java/.NET/JS-style VM target. So you can distribute your app with a single binary download, instead of different ones for different Operating Systems.
- rcfox 8y agoDo wasm modules depend on one another? I assumed they were statically linked. It feels like this is more of a wasm "app store".
- mintplant 8y agoWASM dynamic linking: https://webassembly.org/docs/dynamic-linking/ https://webassembly.org/docs/dynamic-linking/ I don't think WAPM has any particular support for this (yet?), though.
- matthewbauer 8y agoI'm working on Nixpkgs support for WASI cross compilation here: https://github.com/NixOS/nixpkgs/pull/56555 https://github.com/NixOS/nixpkgs/pull/56555
- ilaksh 8y agoI brought up some similar ideas more than three years ago. https://github.com/WebAssembly/design/issues/320 https://github.com/WebAssembly/design/issues/320 In case anyone is interested in any of that discussion.
- dmix 8y agoIt'd be cool to see what language the parent library was written in, so you could explore them similar to Github's language based browse feature. Although I guess the whole point, as far as the end user is concerned, is it doesn't matter what language it was written in. But it's still very relevant to people using them as libraries.
- Illniyar 8y agoSo this is a package manager like yum or apt-get, not like npm. Seems reasonable, I guess, though the power of wasm is that it can be run without install, like a webpage. We'll see if this catches on.
- tokyodude 8y agodoes that mean everything installs globallu by default. if there is one thing I love about npm it's that it's local by default and global is the exception.
- v_lisivka 8y agoIs it possible to integrate wapm with web browser as plugin, so web site will be able to request up-to date wasm packages from local store instead of downloading of their own copies?
- sureaboutthis 8y agoAnd so it begins. The building of "tools for tools". The inability of people to work with the fundamentals. A device in search of a problem.
- fiatjaf 8y agoHow do I use that?
- sitapati 8y agoYou guys are missing the forest for the trees. This is truly revolutionary. https://dev.to/jwulf/i-can-t-believe-it-s-webassembly-308c https://dev.to/jwulf/i-can-t-believe-it-s-webassembly-308c
- intertextuality 8y agoThen submit that to HN, showcasing what wapm can really do. Instead, people here are rightly cynical. I do not want another private company managing something as fundamental as a package registry. I want them to answer my question about how they are actually going to fund themselves. Especially with the dubious decision to pay for office space in San Francisco.
- sitapati 8y agoDone! I'd fund it through VC investment. It's a market/mind share play aka hosting a community.
- intertextuality 8y agoFault on me for wording it that way. 1) How are you going to earn a profit? Like what is your revenue stream going to be, etc? Not funding. 2) Why do you need an office in downtown San Francisco? Why not work remotely?