12 ms·

WASIX, the Superset of WASI Supporting Threads, Processes and Sockets

210 points by the_duke 3y ago

rapnie 3y ago
In their community repo: > We want to welcome as many developers and companies joining and participating in WASIX. This is a project that aims to be governed by many entities, not just one. > Wazero, Zig, and AssemblyScript could be great additions to the governance team, but any others will be more than welcome. As long as you want to move WASIX forward you will be welcome to chime in and participate in its progress. https://github.com/orgs/wasix-org/discussions/1 https://github.com/orgs/wasix-org/discussions/1
- deleted 3y ago
  [deleted]
wokwokwok 3y ago
So, let me get this straight… WASM is a language independent byte code that runs in browsers, like “assembly but for browsers” that multiple languages can compile to instead of there only being javascript. …but that ran in a browser sandbox, and people wanted to use it in containers, so they invented WASI, which lets WASM assembly code interact with the base system, sidestepping the container of the browser; but it needs a runtime on the host system (not a browser) to run. Now this is WASIX which added more non standard extensions to WASI so that more programs can run on it. We had an entire full stack cross platform runtime (browsers) to run our code in, but instead we now need to get a new set of “low level” runtimes to run the same code for, eg. containers? Sounds a lot like java to me; byte code, runtime, “cross platform binaries”, write once, run anywhere… How is this meaningfully different? The only remarkable difference appears to be you can target it with existing c++ code?
- bmacho 3y ago
  I don't think it is meaningfully different than JVM. Why? Should it be?
- jinwoo68 3y ago
  I think you're basically correct. But wasm isn't tightly coupled with a single programming language, like Java did. Of course, other languages can target JVM too, but there's not much momentum there. With wasm, we have many different implementations written in many languages running in many different platforms. I think it's a good thing.
  - inadequatespace 3y ago
    > not much momentum Scala? Kotlin? To be fair these came into play WAY after the JVM
    - greenyoda 3y ago
      Also Clojure.
    - throwaway894345 3y ago
      These are all languages that were designed expressly to run on the JVM, whereas WASM aspires to support existing languages. I'm not familiar with JVM bytecode (and would love to hear from someone who is), but I'm guessing the JVM bytecode offers less in the way of low-level control over memory allocations and so on which would be required to compile C, C++, Rust, Go, etc programs with reasonable performance characteristics?
      - brabel 3y ago
        JVM bytecode assumes the GC allocates and frees memory, so no, you don't have direct access to memory... though the Java standard library does let you allocate bytes and manage those yourself[1], not sure if you would consider that "more control over memory allocations". JVM bytecode is quite approachable: https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions https://en.wikipedia.org/wiki/List_of_Java_bytecode_instruct... And does look superficially similar to WASM instructions. When WASM gets GC, I have trouble seeing what's fundamentally different between the two, to be honest... even something like (parts of) the Java stdlib will be available to WASM with the WASI API, making the difference too small to distinguish them into different categories IMO. [1] https://blogs.oracle.com/javamagazine/post/creating-a-java-off-heap-in-memory-database https://blogs.oracle.com/javamagazine/post/creating-a-java-o...
- reyqn 3y ago
  And with existing .Net, C, Go, Rust, Lua, Zig, Crystal, D, Elixir, also Java, JavaScript, Kotlin, Lisp, Nim, Pascal, Perl, PHP, Python, R, Ruby code (non-exhaustive list). That's a lot of flexibility
  - pjmlp 3y ago
    Just like the CLR in 2001!
    - deleted 3y ago
      [deleted]
    - doodlesdev 3y ago
      So here's the thing, CLR is still here (now called CoreCLR as of the move to .NET Core), how does it differ technically from WASM/WASI, and why aren't we using it instead? I should note I haven't spent much time studying either of them.
      - pjmlp 3y ago
        Mostly because Micro$oft shenanigans, or the fear thereof. As if the VCs baking up all those WASM products are doing it from the warmth of their hearts.
        garganzol 3y ago
        They will do all the mistakes that have to be done. WASM GC is one of them.
        pjmlp 3y ago
        Probably, then again if it was only for C and C++ style workloads, even one more reason to stick to the past. https://en.wikipedia.org/wiki/Architecture_Neutral_Distribution_Format https://en.wikipedia.org/wiki/Architecture_Neutral_Distribut... Just one example out of many others, for C and C++ based workloads.
- pie_flavor 3y ago
  Java is a language with a particular view of the world, and the JVM is a heavyweight runtime for executing that particular view of the world. The core idea of the JVM is great, and WASM executes that without assuming anything about your language, or requiring a great big tightly integrated second-banana-to-the-host-language's-stdlib system library, or anything else Java-y.
  - garganzol 3y ago
    And this why WASM is so great. No opinions on a language, APIs, or anything; just pure execution. Well, WASI and WASIX are APIs but they are not strictly obligatory.
    - pjmlp 3y ago
      Only if one ignores the history of bytecode execution formats all the way back to late 1950's.
      - dpe82 3y ago
        For those of us less informed, care to explain?
  - frant-hartm 3y ago
    When a wasm runtime supports garbage collection and all the already proposed APIs it will be comparable to nodejs or JVM.
    - garganzol 3y ago
      Let's hope that will never happen because it will lead to an imminent WASM death by making it opinionated. I would also suggest to officially separate WASM from any notion of API. So that any working group anywhere in the world could design whatever API they want/need - but it would not wreck the purity of WASM as a result. WASI/WASIX initiatives are good in that regard.
- tlarkworthy 3y ago
  The worse thing is that WASI doesn't support browsers as a target anymore. Edit: this supports browsers!!! Amazing!!!
  - deleted 3y ago
    [deleted]
  - flohofwoe 3y ago
    WASI (or rather POSIX) was never a good fit for the browser application model though. Emscripten did a pretty good job providing an incomplete "POSIX emulation layer" to get things rolling, but for WASM that needs to run in the browser it's better to go with Emscripten's web API mappings.
  - preseinger 3y ago
    wasi was never intended for browsers wasm isn't a browser-only thing
- samsquire 3y ago
  I think we're always looking for the silver bullet that shall make computer software development easy. The following technologies were all silver bullets: * ActiveX, COM Object model, XPCOM * Unified Modeling Language (UML) * Extensible Markup Language (XML) * Computer-aided software engineering (CASE) * Java Virtual Machine * Ionic, React Native, Qt, Gtk, Flutter Something tells me that the problem is bigger than anyone can chew and the abstraction solution is sometimes more complicated than the underlying thing being abstracted. I think WASI and WASIX is an EXTREMELY important API for the future of computing. It's a chance to create a superior operating system runtime interface.
  - lifty 3y ago
    Why does it have the chance to be superior?
    - samsquire 3y ago
      Compiling native application to operating systems that Wasmer is compiled for including the browser and have the code seamlessly work in all of them, I think that's really promising. JVM forces you to write in Java or a JVM targeted language but WASM lets you compile any language that creates a WASM backend.
      - vineyardmike 3y ago
        This > JVM forces you to write in Java or a JVM targeted language And this > WASM lets you compile any language that creates a WASM backend Are basically the same thing. There’s no fundamental reason that all these languages couldn’t target JVM byte code instead of WASM. It never happened, not nearly as much as WASM backends are, but it could’ve.
        ShroudedNight 3y ago
        It seems that NIH also manifests generationally: ? -> CORBA -> RMI -> SOAP -> REST -> GraphQL? ? -> SOA -> Microservices ? -> EDI / X12 -> XML -> JSON p-code -> ? -> JVM -> WASM
- postalrat 3y ago
  Existing c++, zig, rust, etc isn't the goal, it's the start.
- pjmlp 3y ago
  Got to get all that VC money, while trying to resell old ideas.
- deleted 3y ago
  [deleted]
- throwaway894345 3y ago
  The JVM expects that languages will conform to its bytecode specification, which is designed first and foremost for Java. WASM's bytecode specification is designed to support the existing ecosystem of languages including Rust, C, C++, Go, etc which can't easily target the JVM. Relatedly, the body which manages the JVM specification is comprised of major Java vendors and users; they don't represent developers of other languages even if the JVM nominally supports a few of them.
  - kgeist 3y ago
    >WASM's bytecode specification is designed to support the existing ecosystem of languages including Rust, C, C++, Go, etc which can't easily target the JVM Microsoft's .NET allowed you to compile C++ to pure CIL (.NET's bytecode) and run it inside a VM since the early 2000's. It wasn't very popular so it was deprecated. People compare WASM to JVM, but to me it looks like a more lightweight reinvention of Microsoft's CLR (Common Language Runtime) which was designed to support multiple languages (C#, C++, VB) from day 1.
    - TUSF 3y ago
      > Microsoft's .NET There's your problem. Sure, .NET is technically cross-platform, but that's a relatively recent development, and WASM has runtimes that let it run in most environments, even microcontrollers. Also, looks like C++/CLI had conflicting syntax with standard C++, so it might not have worked on some applications without some patches.
- stephc_int13 3y ago
  >The only remarkable difference appears to be you can target it with existing c++ code? That changes everything. Coupling the VM and the language was IMHO a huge mistake. Wasm + WASI(X) + WebGPU is what the Java VM should have been. I guess we were not ready at the time.
  - pjmlp 3y ago
    We were ready plenty of times, Java wasn't the first.
- paulddraper 3y ago
  > Sounds a lot like java to me; byte code, runtime, “cross platform binaries”, write once, run anywhere… 100% WASM is "Java byte code for the 21st century." > How is this meaningfully different? It's lower level, making it a lighter, more flexible target platform for a variety of languages. No classes, typing, GC (yet), required runtime, etc. Whereas Java byte code is really only meant for Java (and variants -- Groovy, Clojure, Scala, Kotlin)
  - sroussey 3y ago
    It’s different from the browser sandbox in that it is targeting servers not clients. That said, you can run a large subset in the browser.
  - pjmlp 3y ago
    So basically CLR in 2001.
    - paulddraper 3y ago
      Basically. A key difference is that it's actually gaining traction on multiple platforms. IDK if CLR tried and failed to do that, or just didn't try.
      - pjmlp 3y ago
        Mono, Xamarin, Project Rotor,...
- moonchrome 3y ago
  Every thread about WASM is "this is just JVM". No it's not. If you want to know how it's meaningfully different - go look look high level overview of JVM and WASM - and if you still don't get it - then you're probably not the target audience for either and you should just be using whatever the tool stack it's targeting them. A bus and a cargo truck are not the same just because they have similar wheel base/size and abstractly do the same thing.
  - jeroenhd 3y ago
    To me WASM just seems like the JVM without special OOP instructions. It never started out that way, but it certainly is ending up that way. Everything that made WASM stand out from the traditional bytecode VM is being patched back in through extensions like WASI and now WASIX. I was originally excited for WASM because it started out as a fully sandboxed system. No I/O other than flat memory buffers, so no chance whatsoever of escaping its sandbox. Now WASIX introduces networking, file system I/O, mutltithreading, forking, even TTY support. It's not as powerful as the JVM yet, but give it a few years and I'm sure you can use them interchangeably in a few years.
    - ShroudedNight 3y ago
      This reminds me of the old adage that the most secure computer is one that has never been, and can never connect to anything. It's also rather limited in its usefulness.
      - jeroenhd 3y ago
        The limitations are what made WASM attractive to me. If I want a fast component to re-encode images, or to render a 3D scene, or to analyze and transform data, I don't want to have to set up a network namespace, alternate file system roots, and all the other annoying stuff that comes with sandboxing. WASM not being able to do or access anything you don't manually provide allows for quickly and easily hooking in executable code without the normal risks. Add enough extensions to the runtime everyone uses and you end up with another JVM. The JVM is very good and so is the .NET runtime and any other bytecode runtime you can think of, but the distinctions start to fade as more features get tacked onto WASM.
- vbezhenar 3y ago
  Yes, it is "JVM reimagined". It has very little with browsers, though. IMO this is wrong direction. Better direction is to move into browser. Add arch/wasm to linux kernel. Compile it to wasm. Compile systemd to wasm, glibc or musl, with docker in the end. Now you can run linux in your browser. You could run it before, but this way makes it as native to browser as possible. Probably will be very fast and suitable for work. Now with docker ask people to compile their software to wasm. Now you can run postgres and kubernetes in your Chrome tab. All you need is some escape hatch for network ingress/egress which is not hard to implement. May be implement virtio-gpu with webgpu and you can easily run full-fledged Gnome in your browser tab. Not hard to sync your disk to S3, so you can open this VM from any computer and run it locally. Basically VM in your browser, cross-platform, cross-architectured.
  - laxd 3y ago
    > Now you can run postgres and kubernetes in your Chrome tab. This is what the world has come to.
  - paulddraper 3y ago
    Sooo.....a VM?
    - deleted 3y ago
      [deleted]
- deleted 3y ago
  [deleted]
- pookha 3y ago
  I think it's more about streamlining application deployments and removing the containerization hurdle...The dream seems to be a universal byte-code (WASM) vm that would open the door so that Python, Node, Java 58LTS, Rust, C or freaky Lua scripts could run on the same runtime -- easily -- without container wrappers that each have to be secured (patches) and with a startup penalty.
- kelp 3y ago
  I think server side WASM has the potential to be the next thing after containers. And yes, lots of echoes of Java's "write once run anywhere" but as lots of people have pointed out, wasm is language agnostic, and many languages have support for it. Since I've been web applications and distributed systems, the evolution has been something like this. 1. Copy files around, into the web or application servers directory. This was simple but isolation, repeatability and testability was hard. The application dependencies were directly tied to the host, so you had to be very careful about your upgrades breaking applications. The lack of isolation is not suitable for multi-tenant environments for security and capacity planning reasons. It's pretty hard to run multiple-applications (1) 2. Things like bundler, and other language specific tools that let you bundle your application dependencies together. Avoids some of the host OS related upgrade issues, but again, no real isolation. 3. VMs. The whole cloud industry started on this. You get true isolation of your whole application at the cost of complexity and weight. Scheduling new VMs is not a fast process because you're copying around or at least booting a whole OS. You still have to manage OS images. There is also generally a tax on memory that you have to pay, so you waste some resources with VMs to get the isolation. Amazon invested in Nitro to avoid paying a lot of this tax. 4. Containers. Lighter than VMs, good enough isolation in a lot of cases, especially inside of an organization. Lower tax than VMs. More testable. But containers are still fairly heavy, startup time can be slowish. So you have some limitations on how fast you can scale up and down in response to load. Also isolation isn't good enough for some workloads. So we see things like firecracker VMs being used to improve isolation. 5. Wasm on the server side. Much lighter than containers. Less memory overhead, way less startup time. Probably faster startup time than even firecracker. So for lambda like workloads you could could have a lot better cold start latency. You probably have good enough isolation that you don't need to resort to things like firecracker to isolate customer workloads. Because of the fast startup time you could probably pack your long tail of workloads even tighter to get better effective utilization out of your server infrastructure. (1) In the early 00s I worked at a place where we wrote our own simple process supervisor and used some tricks with LD_LIBRARY_PATH to let us bundle all application dependencies with the application. So you could run multiple application that needed different libraries on the same host, and upgrade your host without as much worry about host OS library changes breaking our applications. It was a kind of proto-container, without all the other isolation cgroups give you.
- preseinger 3y ago
  wasm targets more than just browsers
jey 3y ago
Sounds and looks like it was fun to implement, but is there a motivating use case? Or is it more for technical/hacking interest?
- garganzol 3y ago
  I think this may become the basis of an operating environment where you do not have to concentrate on "how to run" something. Instead, you get straight to the job of deciding "what to run" so you can reap the benefits in no time without all the devops and SaaS drama. The closest contemporary thing is fly.io, but recently they became less strategic and more opinionated. Another attempt is sandstorm.io but it was abandoned by the original authors. So... maybe WASM and the initiatives like WASIX will give it to us?
  - dboreham 3y ago
    If only someone had built this 25 years ago...
  - rapnie 3y ago
    Looks like Sandstorm still has the original founders on the FOSS project they now are, just not as a company. Both co-founders working for Cloudflare now: https://sandstorm.io/about https://sandstorm.io/about
    - paulgb 3y ago
      Ideas from Sandstorm also live on in Cloudflare's Durable Objects: > We don't have any current plans, but... I was the co-founder of Sandstorm.io before going to Cloudflare, and Durable Objects are very much inspired by parts of Sandstorm's design. So yeah, I've absolutely thought about it. ;) https://news.ycombinator.com/item?id=24619148 https://news.ycombinator.com/item?id=24619148
    - ocdtrekkie 3y ago
      The about page is very not updated, FWIW. I started a PR and then did not finish it. About every other week someone looks at it and gleans incorrect insight from it, so I really should finish that PR... Most contributions presently come from outside the original Sandstorm team, but some significant features and refactors have come out since the company shut down from former employees too.
fofoz 3y ago
Threads and forking in the browser... > As an example, we have created wasmer-web , which basically showcases that any WASIX program published to Wasmer (including those with threads and forking!) works also with Wasmer running on the browser.
- syrusakbary 3y ago
  Here's wasmer-web being deployed: https://wasmer.sh/ https://wasmer.sh/
dboreham 3y ago
[flagged]
ntonozzi 3y ago
How do they get networking and sockets working in the browser? As far as I know, no browsers allow raw network access. Some apps can get around this by using things like a websocket proxy, but this relies on routing your network traffic through a third party. It'd be great to see this solved and to let users host ephemeral network servers from their browsers.
- pcstl 3y ago
  WASI is not meant for browsers, it's for using WebAssembly as platform-independent bytecode in non-browser environments.
  - skybrian 3y ago
    The article claims they got it working in the browser, but not how.
    - dynamitebud 3y ago
      That'll be added using the @wasmer/wasi npm package. We'll be updating that soon
      - mrkstu 3y ago
        So could one use this to run a fully socket capable Jupyter notebook? There are some infosec dashboards I would like to build, but not having the ability to actually run socket capable code killed the interesting aspect of doing it in the browser.
        the8472 3y ago
        Browsers don't have TCP/UDP/etc. sockets, so a WASI impl running in one can't provide that either.
        skybrian 3y ago
MrStonedOne 3y ago
[dead]
skybrian 3y ago
I'm wondering about platform limitations. A web browser doesn't actually let you listen on an arbitrary socket, so how does that work? The supposed specification is more of an API reference and I don't really trust it to tell the developer what you can really do that works well on all platforms. It doesn't document platform-specific limitations, but I don't see it documented anywhere else either. I also wonder about copying Unix semantics for fork and exec in a supposedly platform-independent spec. It seems like that won't work equally well everywhere.
- flohofwoe 3y ago
  WASI (and in extension this WASIX thing) is mainly for running WASM *outside* the browser, basically what POSIX is for native code.
  - skybrian 3y ago
    Yes, understood, but the article talks about running it in a browser.
- garganzol 3y ago
  Easy as 1-2-3. 'sock_listen' will return ENOPERM or something like that. The app can then decide by itself what to do - either bail out with an error message or continue to work with a somewhat restricted functionality.
  - skybrian 3y ago
    “WASIX calls can be stubbed out in a browser” isn’t really living up to the promise of “running seamlessly in a browser.” That’s closer to “not working” than “working.”
    - anticensor 3y ago
      More like "There is no browser I/O port corresponding to your desired WASIX call". ISA compatibility and I/O compatibility do not imply each other. As demonstrated countless times during the early days of the PC.
nynx 3y ago
Defeats the point of WASI—WASIX seems not to be capabilities based
- garganzol 3y ago
  I do not see a problem in that. You can always restrict the capability by not exporting a particular API function, or just by returning ENOPERM. It's the job of an operating environment to decide which permissions are available.
- boricj 3y ago
  WASIX seems to be basically POSIX for WASM (POSIX/WASIX). It is probably an explicit design goal to be able to run POSIX-compliant programs on it. The fact that POSIX is 1970s legacy, rusted beyond salvaging and an extremely poor fit for modern system designs and expectations is another problem.
  - nsonha 3y ago
    What would be a modern alternative?
    - boricj 3y ago
      The only other equivalent API in broad use today that I'm aware of would be the Kernel32.dll subset of Win32 [1]. Not that it's perfect (so much accumulated cruft) or what I would call modern (still has a lot of ambient authority baked-in) by any means, but the important parts are mostly handle-based (unlike PID-based POSIX process management functions for example). I happen to like the design of the syscall layer of Fuchsia's Zircon kernel [2], but it's not a full substitute for POSIX (notably, file and network I/O are built as userspace features of top of IPC channels, they are not kernel concepts). [1] https://www.geoffchappell.com/studies/windows/win32/kernel32/api/index.htm https://www.geoffchappell.com/studies/windows/win32/kernel32... [2] https://fuchsia.dev/fuchsia-src/reference/syscalls https://fuchsia.dev/fuchsia-src/reference/syscalls
c-linkage 3y ago
Why is it that when a new technology comes along for executing code, the first thing people do is say "this is great, but it would be awesome if it ran my 40 year old code without modification"?
- mastax 3y ago
  The value of new code anyone can write in a reasonable amount of time pales in comparison to the value of all the code which has already been written.
- suprfsat 3y ago
  Why is it that people want to continue using tools that work to solve problems no matter how hard this industry tries to leave them in the dust?
- deleted 3y ago
  [deleted]
cmeacham98 3y ago
I notice they specifically use neverssl.com in their demo - is TLS too strenuous or not yet possible for some reason?
- syrusakbary 3y ago
  Because there are not pre-approved certificates in WASIX, you will need to pass your own certificates or trust the ones that the website provides you. In any case, you can access any SSL website with curl -k flag: wasmer run --net curl/curl -- -k https://google.com
pajko 3y ago
Isn't this what PNaCl was essentially about? Providing an API and some kind of VM (llvm) to implement the API on top of it?
higherhalf 3y ago
Excited for threads, sockets, futexes, but not excited for fork, signals and setjmp. They didn't include POSIX stuff like shared memory, select(2), unix sockets, and had to draw a line somewhere. Why include error-prone cruft?
- mananaysiempre 3y ago
  Plain fork (as opposed to vfork) isn’t really error-prone, it’s arguably less error-prone than threads. (It forces bad resource accounting, like most CoW mechanisms, but that’s an orthogonal issue.) Signals and setjmp... have their problems, admittedly, but on a restricted VM like WASM are AFAICT unimplementable on top of other things. So if you want unwinding, you’ve got to have an equivalent to setjmp (or throw / catch) if you want asynchronous interrupts, you’ve got to have an equivalent to signals.
  - jcranmer 3y ago
    > So if you want unwinding, you’ve got to have an equivalent to setjmp (or throw / catch) Unwinding is a somewhat cleaner interface than setjmp/longjmp, and it's a useful primitive in its own right, especially if you can attach a notion of a stack trace to the unwind primitive. At a low-level language primitive level, unwind can be seen essentially as a combination of a mechanism to have two distinct exit points from a function call (the normal and exceptional return addresses) and an intrinsic that causes the function to return via its exceptional return address instead of the normal return address. > if you want asynchronous interrupts, you’ve got to have an equivalent to signals. ... not really. There's a few different kind of signals. Synchronous processor signals (e.g., SIGSEGV, SIGFPE) could be supported via something akin to unwind rather than signal handlers (and this is essentially how they work on Windows). The asynchronous events can instead be supported by basically having some primitive notion of an event loop, dropping those signals as events in the event loop. Signals already interoperate pretty sketchily with event loops, and many signal handlers for asynchronous events already tend to boil down to "just dump this as an event in the event loop", so you're left with a pretty sketchy work around to do what you actually wanted to do. Interfaces like fork and POSIX signals are pretty notorious for being generally wrong ways to do what you actually wanted to do; if you're making a new OS ABI, there's absolutely no reason to include them, especially because they can be hard to emulate on some OSes, e.g., Windows or Fuchsia (which lacks POSIX signals altogether).
siddiqi123 3y ago
I honestly feel it’s good for creating plug-in systems, but it offers app-level portability, unlike docker which provides system-level portability
samsquire 3y ago
I probably should commit to Go, Rust or Erlang but I want to create a model of multithreading that is easy to understand and not error prone. Multithreaded WASM is awesome. My dream is a just in time compiler, performant, simple, easy, scalable, safe multithreaded runtime. I use Java, Rust and C for my multithreaded programs.
- Philpax 3y ago
  Check out Lunatic https://lunatic.solutions/ https://lunatic.solutions/
ori_b 3y ago
This seems pretty inherently difficult to port or to run safely. Things like fuetxes will only work directly on Linux and OpenBSD, for example. From what I recall, OSX has some undocumented futex-like-but-not-quite ulock, and FreeBSD has yet another incompatible tool. Fork is not a windows-native primitive, and while cygwin may have implemented emulation, the performance sucks. Pthreads is in a similar situation. So, given that this will mainly target linux well, I'm not sure what advantages it brings over native code. It mostly feels like native code with extra steps.
- tyingq 3y ago
  This seemed like a reasonable overview of "futex-like" functionality on things other than Linux: https://shift.click/blog/futex-like-apis/ https://shift.click/blog/futex-like-apis/ As you say, it's not straightforward.
- ComputerGuru 3y ago
  WTF locks under C++ and Parking Lot under rust are great low level futex-esque cross-platform abstractions/implementations. They originate from the WebKit codebase (unless it was Blink but I think they predate the fork). They run under Windows, Linux, the various BSDs, and macOS.
5- 3y ago
so, aside from the obvious 'embrace, extend, extinguish' angle, why was this not part of wasi proper? it does seem like wasi has been stuck, in the outside view, for quite a while. would welcome any comments from wasi/wasix insiders.
- dbfa 3y ago
  WASI has been in Preview1 for a couple of years now, indeed. However, Preview2 is well underway and the initial implementation is scheduled to land in wasmtime this June. https://github.com/bytecodealliance/wasmtime/issues/6370 https://github.com/bytecodealliance/wasmtime/issues/6370
  - tokera 3y ago
    preview2 still need these to be added: - full Berkley socket support - sub-process spawning, waiting - forking - TTY support - pipes - event streams - DNS resolving - longjmp/setjmp without these we can't compile our favorite apps in our favorite languages lets help get these into preview2 so that WASIX is not needed anymore but until then the community needs an interim solution that has long term support - that is WASIX's complimentary goal.
- tokera 3y ago
  (WASIX insider) lets reflect on this - if this was done as pull requests to `preview1` then would it have been accepted?... it would have received rejection comments around even the idea of these particular POSIX compatible syscalls. WASIX should be seen as a preview 1.5 rather than something new, just extensions to WASI that add the missing bits that are badly needed today to actual compile useful apps from various code bases. at a minimum this is an interim solution so that app developers can just start compiling their code bases into WASM and we can all get moving forward together - no one wants a fork here. of course the WASI maintainers are reading all this and hopefully this will be a wake up call to add these missing syscalls as a priority also into preview2 so everyone benefits - then WASIX can die and its purpose is served... but until then WASIX will get long term support so the community can proceed literally today onwards.
benatkin 3y ago
Interesting, they're trying to base their credibility on Docker because they had a falling out with the official WASM community. They also quote an old tweet from the founder of Docker, and I can't see any new tweets about Wasmer from him: https://twitter.com/search?q=wasmer%20solomonstre&src=typed_query&f=live https://twitter.com/search?q=wasmer%20solomonstre&src=typed_... He tweeted about a competitor to WAPM, a Wasmer product: https://twitter.com/solomonstre/status/1584817684172005376 https://twitter.com/solomonstre/status/1584817684172005376 It seems likely that shykes moved along from Wasmer along with many others. If there were a time for him to indicate he's still emotionally invested in Wasmer, this would be it!
- brabel 3y ago
  What exactly is your point? Wasmer is doing some pretty good work on top of WASM... There are some pretty neat stuff they have developed, mostly open source as far as I know... why should anyone care who is "moving away" from them?
  - deleted 3y ago
    [deleted]
- epilys 3y ago
  It's worse than that in reality. The runtime is supported by the great wasmer employees. The rest of wasmer is in a constant state of bullshitting themselves and the public. Look what year they started, how many years it has been since then, how many products they have launched since, how many weird and sociopathic and narcissistic displays the company has on public record. Also, of course you can ask other former employees and they'd tell you the same: work culture is toxic. In fact the list of ex-employees is huge because of this... I'm thankful I quit fast enough. If any throwaway account replies to me, you'd know who it is.
shp0ngle 3y ago
Not directly related to this thread, but I was recently unpleasantly surprised how under-defined and undocumented is WASI. I was looking into some detail of a WASI call and there is… basically nothing. Also the latest version of the docs is called something like “snapshot3” or whatever. I thought “wait this is it? that’s the whole definition”? apparently it is. edit: here. The actual latest definition is in "legacy/preview1". Yes that is the latest WASI definition. https://github.com/WebAssembly/WASI/blob/main/legacy/preview1/docs.md https://github.com/WebAssembly/WASI/blob/main/legacy/preview... What do the inputs mean? How should it behave in some extreme conditions? Eh YOLO who knows. All implementation dependent. This document is literally all there is. That's it, that's the WASI definition.
- yuri91 3y ago
  I agree. I implemented a subset of WASI, and the best resource was an autogenerated C header with function signatures and very terse comments, buried in the WASI SDK repo.
  - martindevans 3y ago
    I'm also currently implementing a subset of WASI, would you mind sharing a link to that file?
    - yuri91 3y ago
      Actually, it was in wasi-libc: https://github.com/WebAssembly/wasi-libc/blob/main/libc-bottom-half/headers/public/wasi/api.h https://github.com/WebAssembly/wasi-libc/blob/main/libc-bott...
- sunfish 3y ago
  I work on WASI, and completely agree. A lot of things are not clearly documented right now, and we're working on fixing it.
  - titzer 3y ago
laerus 3y ago
Oh is these guys again that tried to trademark the Webassembly name.
- syrusakbary 3y ago
  https://wasmer.io/posts/wasmer-and-trademarks https://wasmer.io/posts/wasmer-and-trademarks
preseinger 3y ago
it is important to point out that wasmer is an organization that is not in good standing with the wasm/wasi community until that is resolved, their products should probably not be viewed in good faith
- waynr 3y ago
  How involved are you in the WASM/WASI community? In your view, what would it take to get the not-in-good-standing issue resolved? What do you get out of shitposting Wasmer when they announce an initiative they are excited about? (I'm mostly trying to understand your psychology, not whether or if you have some kind of competing financial interest)
  - preseinger 3y ago
    you don't get to classify my comments as "shitposting" i'm summarizing well-understood opinions of the ecosystem
  - preseinger 3y ago
    > In your view, what would it take to get the not-in-good-standing issue resolved? probably the most effective way would be for the organization to release a statement abdicating their previous controversial positions, and for the CEO to resign
- circuit10 3y ago
  Can you explain more?
  - circuit10 3y ago
    Is it this? https://wasmer.io/posts/wasmer-and-trademarks-extended https://wasmer.io/posts/wasmer-and-trademarks-extended
    - preseinger 3y ago
      that's the beginning of it, basically, yes
yuri91 3y ago
I am curious about the browser compatibility layer. Does it support networking? (And if so, is it using a proxy to connect to the internet or is it using a virtualized network layer that stays in the browser?) How is fork implemented? (Asyncify all the code?) I was looking at the Github org page (https://github.com/wasix-org https://github.com/wasix-org), but I can't figure out where this stuff is.
- preseinger 3y ago
  wasi is wasm applied outside of the browser it's basically nonsensical to try to adapt wasi in the browser
  - skybrian 3y ago
    They claim they did it. That's what we're talking about. Did you read the article?
    - preseinger 3y ago
      i'm sure they did it, that's fine my point is that this is a nonsensical thing to do
      - tokera 3y ago
        its only nonsense from ones perspective... from another perspective it makes a lot of sense - if one wants to create a fully sandboxed browser terminal (benefits of which are huge). then compiling bash to WASM goes a long way as who wants to rewrite all that terminal code anyway? https://wasmer.sh/ https://wasmer.sh/
        preseinger 3y ago
        syrus sockpuppet be gone
        3y ago
nneonneo 3y ago
Worth remembering: Wasm on browsers is sandboxed by origin and can't do anything the JS cannot already do. However, Wasm on computers with a full set of syscalls available is isomorphic to native code. Worse, because programs in C/C++ and other unsafe languages can compile to Wasm, any security bugs in those codebases will be present when compiled to Wasm. Wasm programs can still have buffer overflows, use-after-frees, and all the other fun memory safety issues; threads will add data races and memory ordering bugs to the mix. The only benefit of Wasm in this space seems to be that you don't have to recompile your programs for a new architecture - kind of like a lower-level Java. But as soon as your program needs to interact with OS features or existing native libraries, this benefit is going to go out the window...
- sitkack 3y ago
  You are right about mostly everything except all the flaws of C/C++ making it into Wasm. Wasm still has CFI (Control Flow Integrity) and if you don't take the address of a stack variable, no C stack entry will be made, so much of the stack is invisible to user code (C/C++ code). You can still corrupt memory, you can't implement ROP or subvert control flow in a non data-dependent way.
  - tokera 3y ago
    rust compiles to WASM as-well without those flaws. the fact one even has a choice is the whole point
mikewarot 3y ago
The whole point of WASM was to provide fine grained capability based security in a portable environment. This made it safer than previous sandboxes, specifically because your code couldn't just open a random file in the host environment. I think this is a very bad idea, and should be rejected. If not... we'll just take another 10 years until capabilities show up again, continuing the Sisyphean cycle.
- tokera 3y ago
  Twitter Community Notes: * WASIX does not allow one to open a random file in the host environment, in fact WASIX changed absolutely nothing in file system ABI's at all from preview1 - it just added other extensions. * WASIX has not removed the capabilities model and instead extended it to support Berkley sockets - i.e. there are capabilities defined for the new socket operations. * WASIX is fully sandboxed, meaning the network is completely virtual and can be restricted as much as one likes * TTY and forking is also fully sandboxed. * You can't reject the Internet

210 points by the_duke 3y ago

In their community repo: > We want to welcome as many developers and companies joining and participating in WASIX. This is a project that aims to be governed by many entities, not just one. > Wazero, Zig, and AssemblyScript could be great additions to the governance team, but any others will be more than welcome. As long as you want to move WASIX forward you will be welcome to chime in and participate in its progress. https://github.com/orgs/wasix-org/discussions/1 https://github.com/orgs/wasix-org/discussions/1

deleted 3y ago
[deleted]

wokwokwok 3y ago

So, let me get this straight… WASM is a language independent byte code that runs in browsers, like “assembly but for browsers” that multiple languages can compile to instead of there only being javascript. …but that ran in a browser sandbox, and people wanted to use it in containers, so they invented WASI, which lets WASM assembly code interact with the base system, sidestepping the container of the browser; but it needs a runtime on the host system (not a browser) to run. Now this is WASIX which added more non standard extensions to WASI so that more programs can run on it. We had an entire full stack cross platform runtime (browsers) to run our code in, but instead we now need to get a new set of “low level” runtimes to run the same code for, eg. containers? Sounds a lot like java to me; byte code, runtime, “cross platform binaries”, write once, run anywhere… How is this meaningfully different? The only remarkable difference appears to be you can target it with existing c++ code?

bmacho 3y ago
I don't think it is meaningfully different than JVM. Why? Should it be?
jinwoo68 3y ago
I think you're basically correct. But wasm isn't tightly coupled with a single programming language, like Java did. Of course, other languages can target JVM too, but there's not much momentum there. With wasm, we have many different implementations written in many languages running in many different platforms. I think it's a good thing.
- inadequatespace 3y ago
  > not much momentum Scala? Kotlin? To be fair these came into play WAY after the JVM
  - greenyoda 3y ago
    Also Clojure.
  - throwaway894345 3y ago
    These are all languages that were designed expressly to run on the JVM, whereas WASM aspires to support existing languages. I'm not familiar with JVM bytecode (and would love to hear from someone who is), but I'm guessing the JVM bytecode offers less in the way of low-level control over memory allocations and so on which would be required to compile C, C++, Rust, Go, etc programs with reasonable performance characteristics?
    - brabel 3y ago
      JVM bytecode assumes the GC allocates and frees memory, so no, you don't have direct access to memory... though the Java standard library does let you allocate bytes and manage those yourself[1], not sure if you would consider that "more control over memory allocations". JVM bytecode is quite approachable: https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions https://en.wikipedia.org/wiki/List_of_Java_bytecode_instruct... And does look superficially similar to WASM instructions. When WASM gets GC, I have trouble seeing what's fundamentally different between the two, to be honest... even something like (parts of) the Java stdlib will be available to WASM with the WASI API, making the difference too small to distinguish them into different categories IMO. [1] https://blogs.oracle.com/javamagazine/post/creating-a-java-off-heap-in-memory-database https://blogs.oracle.com/javamagazine/post/creating-a-java-o...
reyqn 3y ago
And with existing .Net, C, Go, Rust, Lua, Zig, Crystal, D, Elixir, also Java, JavaScript, Kotlin, Lisp, Nim, Pascal, Perl, PHP, Python, R, Ruby code (non-exhaustive list). That's a lot of flexibility
- pjmlp 3y ago
  Just like the CLR in 2001!
  - deleted 3y ago
    [deleted]
  - doodlesdev 3y ago
    So here's the thing, CLR is still here (now called CoreCLR as of the move to .NET Core), how does it differ technically from WASM/WASI, and why aren't we using it instead? I should note I haven't spent much time studying either of them.
    - pjmlp 3y ago
      Mostly because Micro$oft shenanigans, or the fear thereof. As if the VCs baking up all those WASM products are doing it from the warmth of their hearts.
      - garganzol 3y ago
        They will do all the mistakes that have to be done. WASM GC is one of them.
        pjmlp 3y ago
        Probably, then again if it was only for C and C++ style workloads, even one more reason to stick to the past. https://en.wikipedia.org/wiki/Architecture_Neutral_Distribution_Format https://en.wikipedia.org/wiki/Architecture_Neutral_Distribut... Just one example out of many others, for C and C++ based workloads.
pie_flavor 3y ago
Java is a language with a particular view of the world, and the JVM is a heavyweight runtime for executing that particular view of the world. The core idea of the JVM is great, and WASM executes that without assuming anything about your language, or requiring a great big tightly integrated second-banana-to-the-host-language's-stdlib system library, or anything else Java-y.
- garganzol 3y ago
  And this why WASM is so great. No opinions on a language, APIs, or anything; just pure execution. Well, WASI and WASIX are APIs but they are not strictly obligatory.
  - pjmlp 3y ago
    Only if one ignores the history of bytecode execution formats all the way back to late 1950's.
    - dpe82 3y ago
      For those of us less informed, care to explain?
- frant-hartm 3y ago
  When a wasm runtime supports garbage collection and all the already proposed APIs it will be comparable to nodejs or JVM.
  - garganzol 3y ago
    Let's hope that will never happen because it will lead to an imminent WASM death by making it opinionated. I would also suggest to officially separate WASM from any notion of API. So that any working group anywhere in the world could design whatever API they want/need - but it would not wreck the purity of WASM as a result. WASI/WASIX initiatives are good in that regard.
tlarkworthy 3y ago
The worse thing is that WASI doesn't support browsers as a target anymore. Edit: this supports browsers!!! Amazing!!!
- deleted 3y ago
  [deleted]
- flohofwoe 3y ago
  WASI (or rather POSIX) was never a good fit for the browser application model though. Emscripten did a pretty good job providing an incomplete "POSIX emulation layer" to get things rolling, but for WASM that needs to run in the browser it's better to go with Emscripten's web API mappings.
- preseinger 3y ago
  wasi was never intended for browsers wasm isn't a browser-only thing
samsquire 3y ago
I think we're always looking for the silver bullet that shall make computer software development easy. The following technologies were all silver bullets: * ActiveX, COM Object model, XPCOM * Unified Modeling Language (UML) * Extensible Markup Language (XML) * Computer-aided software engineering (CASE) * Java Virtual Machine * Ionic, React Native, Qt, Gtk, Flutter Something tells me that the problem is bigger than anyone can chew and the abstraction solution is sometimes more complicated than the underlying thing being abstracted. I think WASI and WASIX is an EXTREMELY important API for the future of computing. It's a chance to create a superior operating system runtime interface.
- lifty 3y ago
  Why does it have the chance to be superior?
  - samsquire 3y ago
    Compiling native application to operating systems that Wasmer is compiled for including the browser and have the code seamlessly work in all of them, I think that's really promising. JVM forces you to write in Java or a JVM targeted language but WASM lets you compile any language that creates a WASM backend.
    - vineyardmike 3y ago
      This > JVM forces you to write in Java or a JVM targeted language And this > WASM lets you compile any language that creates a WASM backend Are basically the same thing. There’s no fundamental reason that all these languages couldn’t target JVM byte code instead of WASM. It never happened, not nearly as much as WASM backends are, but it could’ve.
      - ShroudedNight 3y ago
        It seems that NIH also manifests generationally: ? -> CORBA -> RMI -> SOAP -> REST -> GraphQL? ? -> SOA -> Microservices ? -> EDI / X12 -> XML -> JSON p-code -> ? -> JVM -> WASM
postalrat 3y ago
Existing c++, zig, rust, etc isn't the goal, it's the start.
pjmlp 3y ago
Got to get all that VC money, while trying to resell old ideas.
deleted 3y ago
[deleted]
throwaway894345 3y ago
The JVM expects that languages will conform to its bytecode specification, which is designed first and foremost for Java. WASM's bytecode specification is designed to support the existing ecosystem of languages including Rust, C, C++, Go, etc which can't easily target the JVM. Relatedly, the body which manages the JVM specification is comprised of major Java vendors and users; they don't represent developers of other languages even if the JVM nominally supports a few of them.
- kgeist 3y ago
  >WASM's bytecode specification is designed to support the existing ecosystem of languages including Rust, C, C++, Go, etc which can't easily target the JVM Microsoft's .NET allowed you to compile C++ to pure CIL (.NET's bytecode) and run it inside a VM since the early 2000's. It wasn't very popular so it was deprecated. People compare WASM to JVM, but to me it looks like a more lightweight reinvention of Microsoft's CLR (Common Language Runtime) which was designed to support multiple languages (C#, C++, VB) from day 1.
  - TUSF 3y ago
    > Microsoft's .NET There's your problem. Sure, .NET is technically cross-platform, but that's a relatively recent development, and WASM has runtimes that let it run in most environments, even microcontrollers. Also, looks like C++/CLI had conflicting syntax with standard C++, so it might not have worked on some applications without some patches.
stephc_int13 3y ago
>The only remarkable difference appears to be you can target it with existing c++ code? That changes everything. Coupling the VM and the language was IMHO a huge mistake. Wasm + WASI(X) + WebGPU is what the Java VM should have been. I guess we were not ready at the time.
- pjmlp 3y ago
  We were ready plenty of times, Java wasn't the first.
paulddraper 3y ago
> Sounds a lot like java to me; byte code, runtime, “cross platform binaries”, write once, run anywhere… 100% WASM is "Java byte code for the 21st century." > How is this meaningfully different? It's lower level, making it a lighter, more flexible target platform for a variety of languages. No classes, typing, GC (yet), required runtime, etc. Whereas Java byte code is really only meant for Java (and variants -- Groovy, Clojure, Scala, Kotlin)
- sroussey 3y ago
  It’s different from the browser sandbox in that it is targeting servers not clients. That said, you can run a large subset in the browser.
- pjmlp 3y ago
  So basically CLR in 2001.
  - paulddraper 3y ago
    Basically. A key difference is that it's actually gaining traction on multiple platforms. IDK if CLR tried and failed to do that, or just didn't try.
    - pjmlp 3y ago
      Mono, Xamarin, Project Rotor,...
moonchrome 3y ago
Every thread about WASM is "this is just JVM". No it's not. If you want to know how it's meaningfully different - go look look high level overview of JVM and WASM - and if you still don't get it - then you're probably not the target audience for either and you should just be using whatever the tool stack it's targeting them. A bus and a cargo truck are not the same just because they have similar wheel base/size and abstractly do the same thing.
- jeroenhd 3y ago
  To me WASM just seems like the JVM without special OOP instructions. It never started out that way, but it certainly is ending up that way. Everything that made WASM stand out from the traditional bytecode VM is being patched back in through extensions like WASI and now WASIX. I was originally excited for WASM because it started out as a fully sandboxed system. No I/O other than flat memory buffers, so no chance whatsoever of escaping its sandbox. Now WASIX introduces networking, file system I/O, mutltithreading, forking, even TTY support. It's not as powerful as the JVM yet, but give it a few years and I'm sure you can use them interchangeably in a few years.
  - ShroudedNight 3y ago
    This reminds me of the old adage that the most secure computer is one that has never been, and can never connect to anything. It's also rather limited in its usefulness.
    - jeroenhd 3y ago
      The limitations are what made WASM attractive to me. If I want a fast component to re-encode images, or to render a 3D scene, or to analyze and transform data, I don't want to have to set up a network namespace, alternate file system roots, and all the other annoying stuff that comes with sandboxing. WASM not being able to do or access anything you don't manually provide allows for quickly and easily hooking in executable code without the normal risks. Add enough extensions to the runtime everyone uses and you end up with another JVM. The JVM is very good and so is the .NET runtime and any other bytecode runtime you can think of, but the distinctions start to fade as more features get tacked onto WASM.
vbezhenar 3y ago
Yes, it is "JVM reimagined". It has very little with browsers, though. IMO this is wrong direction. Better direction is to move into browser. Add arch/wasm to linux kernel. Compile it to wasm. Compile systemd to wasm, glibc or musl, with docker in the end. Now you can run linux in your browser. You could run it before, but this way makes it as native to browser as possible. Probably will be very fast and suitable for work. Now with docker ask people to compile their software to wasm. Now you can run postgres and kubernetes in your Chrome tab. All you need is some escape hatch for network ingress/egress which is not hard to implement. May be implement virtio-gpu with webgpu and you can easily run full-fledged Gnome in your browser tab. Not hard to sync your disk to S3, so you can open this VM from any computer and run it locally. Basically VM in your browser, cross-platform, cross-architectured.
- laxd 3y ago
  > Now you can run postgres and kubernetes in your Chrome tab. This is what the world has come to.
- paulddraper 3y ago
  Sooo.....a VM?
  - deleted 3y ago
    [deleted]
deleted 3y ago
[deleted]
pookha 3y ago
I think it's more about streamlining application deployments and removing the containerization hurdle...The dream seems to be a universal byte-code (WASM) vm that would open the door so that Python, Node, Java 58LTS, Rust, C or freaky Lua scripts could run on the same runtime -- easily -- without container wrappers that each have to be secured (patches) and with a startup penalty.
kelp 3y ago
I think server side WASM has the potential to be the next thing after containers. And yes, lots of echoes of Java's "write once run anywhere" but as lots of people have pointed out, wasm is language agnostic, and many languages have support for it. Since I've been web applications and distributed systems, the evolution has been something like this. 1. Copy files around, into the web or application servers directory. This was simple but isolation, repeatability and testability was hard. The application dependencies were directly tied to the host, so you had to be very careful about your upgrades breaking applications. The lack of isolation is not suitable for multi-tenant environments for security and capacity planning reasons. It's pretty hard to run multiple-applications (1) 2. Things like bundler, and other language specific tools that let you bundle your application dependencies together. Avoids some of the host OS related upgrade issues, but again, no real isolation. 3. VMs. The whole cloud industry started on this. You get true isolation of your whole application at the cost of complexity and weight. Scheduling new VMs is not a fast process because you're copying around or at least booting a whole OS. You still have to manage OS images. There is also generally a tax on memory that you have to pay, so you waste some resources with VMs to get the isolation. Amazon invested in Nitro to avoid paying a lot of this tax. 4. Containers. Lighter than VMs, good enough isolation in a lot of cases, especially inside of an organization. Lower tax than VMs. More testable. But containers are still fairly heavy, startup time can be slowish. So you have some limitations on how fast you can scale up and down in response to load. Also isolation isn't good enough for some workloads. So we see things like firecracker VMs being used to improve isolation. 5. Wasm on the server side. Much lighter than containers. Less memory overhead, way less startup time. Probably faster startup time than even firecracker. So for lambda like workloads you could could have a lot better cold start latency. You probably have good enough isolation that you don't need to resort to things like firecracker to isolate customer workloads. Because of the fast startup time you could probably pack your long tail of workloads even tighter to get better effective utilization out of your server infrastructure. (1) In the early 00s I worked at a place where we wrote our own simple process supervisor and used some tricks with LD_LIBRARY_PATH to let us bundle all application dependencies with the application. So you could run multiple application that needed different libraries on the same host, and upgrade your host without as much worry about host OS library changes breaking our applications. It was a kind of proto-container, without all the other isolation cgroups give you.
preseinger 3y ago
wasm targets more than just browsers

jey 3y ago

Sounds and looks like it was fun to implement, but is there a motivating use case? Or is it more for technical/hacking interest?

garganzol 3y ago
I think this may become the basis of an operating environment where you do not have to concentrate on "how to run" something. Instead, you get straight to the job of deciding "what to run" so you can reap the benefits in no time without all the devops and SaaS drama. The closest contemporary thing is fly.io, but recently they became less strategic and more opinionated. Another attempt is sandstorm.io but it was abandoned by the original authors. So... maybe WASM and the initiatives like WASIX will give it to us?
- dboreham 3y ago
  If only someone had built this 25 years ago...
- rapnie 3y ago
  Looks like Sandstorm still has the original founders on the FOSS project they now are, just not as a company. Both co-founders working for Cloudflare now: https://sandstorm.io/about https://sandstorm.io/about
  - paulgb 3y ago
    Ideas from Sandstorm also live on in Cloudflare's Durable Objects: > We don't have any current plans, but... I was the co-founder of Sandstorm.io before going to Cloudflare, and Durable Objects are very much inspired by parts of Sandstorm's design. So yeah, I've absolutely thought about it. ;) https://news.ycombinator.com/item?id=24619148 https://news.ycombinator.com/item?id=24619148
  - ocdtrekkie 3y ago
    The about page is very not updated, FWIW. I started a PR and then did not finish it. About every other week someone looks at it and gleans incorrect insight from it, so I really should finish that PR... Most contributions presently come from outside the original Sandstorm team, but some significant features and refactors have come out since the company shut down from former employees too.

fofoz 3y ago

Threads and forking in the browser... > As an example, we have created wasmer-web , which basically showcases that any WASIX program published to Wasmer (including those with threads and forking!) works also with Wasmer running on the browser.

syrusakbary 3y ago
Here's wasmer-web being deployed: https://wasmer.sh/ https://wasmer.sh/

dboreham 3y ago

[flagged]

ntonozzi 3y ago

How do they get networking and sockets working in the browser? As far as I know, no browsers allow raw network access. Some apps can get around this by using things like a websocket proxy, but this relies on routing your network traffic through a third party. It'd be great to see this solved and to let users host ephemeral network servers from their browsers.

pcstl 3y ago
WASI is not meant for browsers, it's for using WebAssembly as platform-independent bytecode in non-browser environments.
- skybrian 3y ago
  The article claims they got it working in the browser, but not how.
  - dynamitebud 3y ago
    That'll be added using the @wasmer/wasi npm package. We'll be updating that soon
    - mrkstu 3y ago
      So could one use this to run a fully socket capable Jupyter notebook? There are some infosec dashboards I would like to build, but not having the ability to actually run socket capable code killed the interesting aspect of doing it in the browser.
      - the8472 3y ago
        Browsers don't have TCP/UDP/etc. sockets, so a WASI impl running in one can't provide that either.
        skybrian 3y ago

MrStonedOne 3y ago

[dead]

skybrian 3y ago

I'm wondering about platform limitations. A web browser doesn't actually let you listen on an arbitrary socket, so how does that work? The supposed specification is more of an API reference and I don't really trust it to tell the developer what you can really do that works well on all platforms. It doesn't document platform-specific limitations, but I don't see it documented anywhere else either. I also wonder about copying Unix semantics for fork and exec in a supposedly platform-independent spec. It seems like that won't work equally well everywhere.

flohofwoe 3y ago
WASI (and in extension this WASIX thing) is mainly for running WASM *outside* the browser, basically what POSIX is for native code.
- skybrian 3y ago
  Yes, understood, but the article talks about running it in a browser.
garganzol 3y ago
Easy as 1-2-3. 'sock_listen' will return ENOPERM or something like that. The app can then decide by itself what to do - either bail out with an error message or continue to work with a somewhat restricted functionality.
- skybrian 3y ago
  “WASIX calls can be stubbed out in a browser” isn’t really living up to the promise of “running seamlessly in a browser.” That’s closer to “not working” than “working.”
  - anticensor 3y ago
    More like "There is no browser I/O port corresponding to your desired WASIX call". ISA compatibility and I/O compatibility do not imply each other. As demonstrated countless times during the early days of the PC.

nynx 3y ago

Defeats the point of WASI—WASIX seems not to be capabilities based

garganzol 3y ago
I do not see a problem in that. You can always restrict the capability by not exporting a particular API function, or just by returning ENOPERM. It's the job of an operating environment to decide which permissions are available.
boricj 3y ago
WASIX seems to be basically POSIX for WASM (POSIX/WASIX). It is probably an explicit design goal to be able to run POSIX-compliant programs on it. The fact that POSIX is 1970s legacy, rusted beyond salvaging and an extremely poor fit for modern system designs and expectations is another problem.
- nsonha 3y ago
  What would be a modern alternative?
  - boricj 3y ago
    The only other equivalent API in broad use today that I'm aware of would be the Kernel32.dll subset of Win32 [1]. Not that it's perfect (so much accumulated cruft) or what I would call modern (still has a lot of ambient authority baked-in) by any means, but the important parts are mostly handle-based (unlike PID-based POSIX process management functions for example). I happen to like the design of the syscall layer of Fuchsia's Zircon kernel [2], but it's not a full substitute for POSIX (notably, file and network I/O are built as userspace features of top of IPC channels, they are not kernel concepts). [1] https://www.geoffchappell.com/studies/windows/win32/kernel32/api/index.htm https://www.geoffchappell.com/studies/windows/win32/kernel32... [2] https://fuchsia.dev/fuchsia-src/reference/syscalls https://fuchsia.dev/fuchsia-src/reference/syscalls

c-linkage 3y ago

Why is it that when a new technology comes along for executing code, the first thing people do is say "this is great, but it would be awesome if it ran my 40 year old code without modification"?

mastax 3y ago
The value of new code anyone can write in a reasonable amount of time pales in comparison to the value of all the code which has already been written.
suprfsat 3y ago
Why is it that people want to continue using tools that work to solve problems no matter how hard this industry tries to leave them in the dust?
deleted 3y ago
[deleted]

cmeacham98 3y ago

I notice they specifically use neverssl.com in their demo - is TLS too strenuous or not yet possible for some reason?

syrusakbary 3y ago
Because there are not pre-approved certificates in WASIX, you will need to pass your own certificates or trust the ones that the website provides you. In any case, you can access any SSL website with curl -k flag: wasmer run --net curl/curl -- -k https://google.com

pajko 3y ago

Isn't this what PNaCl was essentially about? Providing an API and some kind of VM (llvm) to implement the API on top of it?

higherhalf 3y ago

Excited for threads, sockets, futexes, but not excited for fork, signals and setjmp. They didn't include POSIX stuff like shared memory, select(2), unix sockets, and had to draw a line somewhere. Why include error-prone cruft?

mananaysiempre 3y ago
Plain fork (as opposed to vfork) isn’t really error-prone, it’s arguably less error-prone than threads. (It forces bad resource accounting, like most CoW mechanisms, but that’s an orthogonal issue.) Signals and setjmp... have their problems, admittedly, but on a restricted VM like WASM are AFAICT unimplementable on top of other things. So if you want unwinding, you’ve got to have an equivalent to setjmp (or throw / catch) if you want asynchronous interrupts, you’ve got to have an equivalent to signals.
- jcranmer 3y ago
  > So if you want unwinding, you’ve got to have an equivalent to setjmp (or throw / catch) Unwinding is a somewhat cleaner interface than setjmp/longjmp, and it's a useful primitive in its own right, especially if you can attach a notion of a stack trace to the unwind primitive. At a low-level language primitive level, unwind can be seen essentially as a combination of a mechanism to have two distinct exit points from a function call (the normal and exceptional return addresses) and an intrinsic that causes the function to return via its exceptional return address instead of the normal return address. > if you want asynchronous interrupts, you’ve got to have an equivalent to signals. ... not really. There's a few different kind of signals. Synchronous processor signals (e.g., SIGSEGV, SIGFPE) could be supported via something akin to unwind rather than signal handlers (and this is essentially how they work on Windows). The asynchronous events can instead be supported by basically having some primitive notion of an event loop, dropping those signals as events in the event loop. Signals already interoperate pretty sketchily with event loops, and many signal handlers for asynchronous events already tend to boil down to "just dump this as an event in the event loop", so you're left with a pretty sketchy work around to do what you actually wanted to do. Interfaces like fork and POSIX signals are pretty notorious for being generally wrong ways to do what you actually wanted to do; if you're making a new OS ABI, there's absolutely no reason to include them, especially because they can be hard to emulate on some OSes, e.g., Windows or Fuchsia (which lacks POSIX signals altogether).

siddiqi123 3y ago

I honestly feel it’s good for creating plug-in systems, but it offers app-level portability, unlike docker which provides system-level portability

samsquire 3y ago

I probably should commit to Go, Rust or Erlang but I want to create a model of multithreading that is easy to understand and not error prone. Multithreaded WASM is awesome. My dream is a just in time compiler, performant, simple, easy, scalable, safe multithreaded runtime. I use Java, Rust and C for my multithreaded programs.

Philpax 3y ago
Check out Lunatic https://lunatic.solutions/ https://lunatic.solutions/

ori_b 3y ago

This seems pretty inherently difficult to port or to run safely. Things like fuetxes will only work directly on Linux and OpenBSD, for example. From what I recall, OSX has some undocumented futex-like-but-not-quite ulock, and FreeBSD has yet another incompatible tool. Fork is not a windows-native primitive, and while cygwin may have implemented emulation, the performance sucks. Pthreads is in a similar situation. So, given that this will mainly target linux well, I'm not sure what advantages it brings over native code. It mostly feels like native code with extra steps.

tyingq 3y ago
This seemed like a reasonable overview of "futex-like" functionality on things other than Linux: https://shift.click/blog/futex-like-apis/ https://shift.click/blog/futex-like-apis/ As you say, it's not straightforward.
ComputerGuru 3y ago
WTF locks under C++ and Parking Lot under rust are great low level futex-esque cross-platform abstractions/implementations. They originate from the WebKit codebase (unless it was Blink but I think they predate the fork). They run under Windows, Linux, the various BSDs, and macOS.

5- 3y ago

so, aside from the obvious 'embrace, extend, extinguish' angle, why was this not part of wasi proper? it does seem like wasi has been stuck, in the outside view, for quite a while. would welcome any comments from wasi/wasix insiders.

dbfa 3y ago
WASI has been in Preview1 for a couple of years now, indeed. However, Preview2 is well underway and the initial implementation is scheduled to land in wasmtime this June. https://github.com/bytecodealliance/wasmtime/issues/6370 https://github.com/bytecodealliance/wasmtime/issues/6370
- tokera 3y ago
  preview2 still need these to be added: - full Berkley socket support - sub-process spawning, waiting - forking - TTY support - pipes - event streams - DNS resolving - longjmp/setjmp without these we can't compile our favorite apps in our favorite languages lets help get these into preview2 so that WASIX is not needed anymore but until then the community needs an interim solution that has long term support - that is WASIX's complimentary goal.
tokera 3y ago
(WASIX insider) lets reflect on this - if this was done as pull requests to `preview1` then would it have been accepted?... it would have received rejection comments around even the idea of these particular POSIX compatible syscalls. WASIX should be seen as a preview 1.5 rather than something new, just extensions to WASI that add the missing bits that are badly needed today to actual compile useful apps from various code bases. at a minimum this is an interim solution so that app developers can just start compiling their code bases into WASM and we can all get moving forward together - no one wants a fork here. of course the WASI maintainers are reading all this and hopefully this will be a wake up call to add these missing syscalls as a priority also into preview2 so everyone benefits - then WASIX can die and its purpose is served... but until then WASIX will get long term support so the community can proceed literally today onwards.

benatkin 3y ago

Interesting, they're trying to base their credibility on Docker because they had a falling out with the official WASM community. They also quote an old tweet from the founder of Docker, and I can't see any new tweets about Wasmer from him: https://twitter.com/search?q=wasmer%20solomonstre&src=typed_query&f=live https://twitter.com/search?q=wasmer%20solomonstre&src=typed_... He tweeted about a competitor to WAPM, a Wasmer product: https://twitter.com/solomonstre/status/1584817684172005376 https://twitter.com/solomonstre/status/1584817684172005376 It seems likely that shykes moved along from Wasmer along with many others. If there were a time for him to indicate he's still emotionally invested in Wasmer, this would be it!

brabel 3y ago
What exactly is your point? Wasmer is doing some pretty good work on top of WASM... There are some pretty neat stuff they have developed, mostly open source as far as I know... why should anyone care who is "moving away" from them?
- deleted 3y ago
  [deleted]
epilys 3y ago
It's worse than that in reality. The runtime is supported by the great wasmer employees. The rest of wasmer is in a constant state of bullshitting themselves and the public. Look what year they started, how many years it has been since then, how many products they have launched since, how many weird and sociopathic and narcissistic displays the company has on public record. Also, of course you can ask other former employees and they'd tell you the same: work culture is toxic. In fact the list of ex-employees is huge because of this... I'm thankful I quit fast enough. If any throwaway account replies to me, you'd know who it is.

shp0ngle 3y ago

Not directly related to this thread, but I was recently unpleasantly surprised how under-defined and undocumented is WASI. I was looking into some detail of a WASI call and there is… basically nothing. Also the latest version of the docs is called something like “snapshot3” or whatever. I thought “wait this is it? that’s the whole definition”? apparently it is. edit: here. The actual latest definition is in "legacy/preview1". Yes that is the latest WASI definition. https://github.com/WebAssembly/WASI/blob/main/legacy/preview1/docs.md https://github.com/WebAssembly/WASI/blob/main/legacy/preview... What do the inputs mean? How should it behave in some extreme conditions? Eh YOLO who knows. All implementation dependent. This document is literally all there is. That's it, that's the WASI definition.

yuri91 3y ago
I agree. I implemented a subset of WASI, and the best resource was an autogenerated C header with function signatures and very terse comments, buried in the WASI SDK repo.
- martindevans 3y ago
  I'm also currently implementing a subset of WASI, would you mind sharing a link to that file?
  - yuri91 3y ago
    Actually, it was in wasi-libc: https://github.com/WebAssembly/wasi-libc/blob/main/libc-bottom-half/headers/public/wasi/api.h https://github.com/WebAssembly/wasi-libc/blob/main/libc-bott...
sunfish 3y ago
I work on WASI, and completely agree. A lot of things are not clearly documented right now, and we're working on fixing it.
- titzer 3y ago

laerus 3y ago

Oh is these guys again that tried to trademark the Webassembly name.

syrusakbary 3y ago
https://wasmer.io/posts/wasmer-and-trademarks https://wasmer.io/posts/wasmer-and-trademarks

preseinger 3y ago

it is important to point out that wasmer is an organization that is not in good standing with the wasm/wasi community until that is resolved, their products should probably not be viewed in good faith

waynr 3y ago
How involved are you in the WASM/WASI community? In your view, what would it take to get the not-in-good-standing issue resolved? What do you get out of shitposting Wasmer when they announce an initiative they are excited about? (I'm mostly trying to understand your psychology, not whether or if you have some kind of competing financial interest)
- preseinger 3y ago
  you don't get to classify my comments as "shitposting" i'm summarizing well-understood opinions of the ecosystem
- preseinger 3y ago
  > In your view, what would it take to get the not-in-good-standing issue resolved? probably the most effective way would be for the organization to release a statement abdicating their previous controversial positions, and for the CEO to resign
circuit10 3y ago
Can you explain more?
- circuit10 3y ago
  Is it this? https://wasmer.io/posts/wasmer-and-trademarks-extended https://wasmer.io/posts/wasmer-and-trademarks-extended
  - preseinger 3y ago
    that's the beginning of it, basically, yes

yuri91 3y ago

I am curious about the browser compatibility layer. Does it support networking? (And if so, is it using a proxy to connect to the internet or is it using a virtualized network layer that stays in the browser?) How is fork implemented? (Asyncify all the code?) I was looking at the Github org page (https://github.com/wasix-org https://github.com/wasix-org), but I can't figure out where this stuff is.

preseinger 3y ago
wasi is wasm applied outside of the browser it's basically nonsensical to try to adapt wasi in the browser
- skybrian 3y ago
  They claim they did it. That's what we're talking about. Did you read the article?
  - preseinger 3y ago
    i'm sure they did it, that's fine my point is that this is a nonsensical thing to do
    - tokera 3y ago
      its only nonsense from ones perspective... from another perspective it makes a lot of sense - if one wants to create a fully sandboxed browser terminal (benefits of which are huge). then compiling bash to WASM goes a long way as who wants to rewrite all that terminal code anyway? https://wasmer.sh/ https://wasmer.sh/
      - preseinger 3y ago
        syrus sockpuppet be gone
        3y ago

nneonneo 3y ago

Worth remembering: Wasm on browsers is sandboxed by origin and can't do anything the JS cannot already do. However, Wasm on computers with a full set of syscalls available is isomorphic to native code. Worse, because programs in C/C++ and other unsafe languages can compile to Wasm, any security bugs in those codebases will be present when compiled to Wasm. Wasm programs can still have buffer overflows, use-after-frees, and all the other fun memory safety issues; threads will add data races and memory ordering bugs to the mix. The only benefit of Wasm in this space seems to be that you don't have to recompile your programs for a new architecture - kind of like a lower-level Java. But as soon as your program needs to interact with OS features or existing native libraries, this benefit is going to go out the window...

sitkack 3y ago
You are right about mostly everything except all the flaws of C/C++ making it into Wasm. Wasm still has CFI (Control Flow Integrity) and if you don't take the address of a stack variable, no C stack entry will be made, so much of the stack is invisible to user code (C/C++ code). You can still corrupt memory, you can't implement ROP or subvert control flow in a non data-dependent way.
- tokera 3y ago
  rust compiles to WASM as-well without those flaws. the fact one even has a choice is the whole point

mikewarot 3y ago

The whole point of WASM was to provide fine grained capability based security in a portable environment. This made it safer than previous sandboxes, specifically because your code couldn't just open a random file in the host environment. I think this is a very bad idea, and should be rejected. If not... we'll just take another 10 years until capabilities show up again, continuing the Sisyphean cycle.

tokera 3y ago
Twitter Community Notes: * WASIX does not allow one to open a random file in the host environment, in fact WASIX changed absolutely nothing in file system ABI's at all from preview1 - it just added other extensions. * WASIX has not removed the capabilities model and instead extended it to support Berkley sockets - i.e. there are capabilities defined for the new socket operations. * WASIX is fully sandboxed, meaning the network is completely virtual and can be restricted as much as one likes * TTY and forking is also fully sandboxed. * You can't reject the Internet