9 ms·
WebAssembly is eating the database?
- nilslice 3y agowhile wasm is getting a ton of attention for its new place in the serverless world, I'm excited about how i may never write another sql stored procedure, or line of pl/sql again!
- neuronicnobody 3y agoWill be interesting to see how interface standards develop across the database ecosystem. I want to write my UDFs once and run them on multiple platforms
- tbm57 3y agotangentially related, but could anybody give me the TLDR on whether increased usage of WASM on the internet is expected to open up a new slew of security vulnerabilities for users? Someone posted on here before that even JS was controversial during its adoption (and now for some users), I can only imagine that some of those concerns are multiplied when you're talking about something with such broad capabilities as WASM
- afiori 3y agoFor now wasm does not have any more risk than obfuscated JS. It is possible that there are a lot of tool to examine JS code for "suspicious code" and that those tools might not work well on WASM. In practice the WASM interpreter in the browser is a functionality that can be (almost) completely polyfilled. Personally I would be way more worries about hardware exploits via WebGL.
- afiori 3y agoAlso it is likely that the browser debugger will be less feturefull for wasm and that future multithreaded support will make obfuscation even easier. But it is probably already possible with JSFuck and webworkers.
- paulgb 3y agoWebAssembly running in the browser runs in an isolated environment whose only access to the outside world is by calling out to JavaScript functions exposed to it when it was initialized. So it doesn’t open up any new APIs. It does get recompiled to bytecode that runs directly on the processor, so in theory there could be vulnerabilities that come up from that, but I think it’s a pretty well-understood surface area at this point.
- folkrav 3y agoI know it's not fundamentally that different from the running arbitrary code we have right now, but something about the idea of indiscriminately running binaries invisibly downloaded over the WWW rubs me the wrong way. Edit: read the first half of my sentence again.
- dymk 3y agoThat’s what we’ve been doing since the dawn of JavaScript, in a VM with a much larger surface area. A simper VM should be quite a bit safer than the status-quo.
- codeflo 3y ago> something about the idea of indiscriminately running binaries invisibly downloaded over the WWW rubs me the wrong way I think that's because the word "binary" has connotations that aren't strictly technically justified. There's no difference between running WASM and running JS (which is usually obfuscated anyway). Both are JITted into machine code, and it's important the JIT is implemented securely. (JIT compilers have been sources of exploits.) The only difference is complexity of the source language, where simpler languages are easier to secure. And WASM is orders of magnitude simpler than JS.
- lumb63 3y agoI have some bad news for you…
- afiori 3y agoI believe wasm will allow for far greater opportunities for code obfustation. You can already do things like http://www.jsfuck.com/ http://www.jsfuck.com/ and even more complex ones, but they would likely have bad performace. With wasm you could do malware-level obfuscation with less drawbacks.
- Jweb_Guru 3y agoWASM's runtime is far simpler than JS's, or even something like the JVM, particularly due to lack of garbage collection. The language was deliberately designed to be straightforward and efficient to JIT compile to a safe virtual machine, e.g. attention was paid to making sure bounds checking could be elided efficiently for constant offsets while being enabled elsewhere. It's also significantly simplified compared to assembly, and lacks functionality like arbitrary jumps that are common sources of vulnerabilities. It's pretty much ideal for the usecase of "I want to run pretty fast code on an untrusted machine" which is the main reason it's taking off. (Obviously, as WASM JIT compilers get more elaborate and it incorporates things like GC'd references, some of the advantages of this design wrt simplicity will disappear, but I anticipate that these kinds of features will mostly not be used in settings like databases).
- malkia 3y agoDart/flutter can now target wasm (unstable), so better gc support is coming (e.g. not gc running from whatever you compiled, but rather the wasm vm would do for you)
- kevingadd 3y agoWASM being used to exploit your PC isn't much of a risk, because it has a really robust security model. It's been used in exploit chains before but my understanding is that the number of times people have found holes in WASM sandboxes is much smaller than the number of times people have found holes in JS runtimes. So it's pretty fit for purpose there. Unfortunately, the design of WASM and common compilers targeting it means that your security is at risk as a user, even though your PC is safe. Applications compiled to WASM have a wide variety of fun security vulnerabilities that have been gone on native targets for decades, which means that if (for example) Gmail moved to WASM, it would now be possible for malicious parties to attack your Gmail tab even though they can't attack your PC. Some examples: * Address space layout randomization is gone - if an attacker gets a write-anwyhere primitive, it will work 100% of the time * Function pointers are densely packed - any possible value within the correct range (0 - function count) is a valid function pointer, as long as the signature matches. Even worse than not having ASLR. * Page protections are gone - all data is mutable, even things that shouldn't be like string constants compiled into the binary * Zero page accesses don't trap - while compilers go out of their way to try and make reading/writing from null pointers fail in WASM, the actual runtime happily allows you to do it. This makes attacks easier to execute because while stray nulls would kill a native application on dereference, in WASM they will just yield a 0 and execution will often continue. * Most stuff is static linked - for native applications, it is common (albeit less common now in our modern era of Electron Hell) to pull in services from the OS and its packages, whether it's zlib or https or whatnot. The WASM runtime model generally offers a very limited set of capabilities in comparison, and there's no equivalent way to dynamically link against vendored packages that get security updates automatically. So every application ships its own ICU (for time zone and localization data), ships its own zlib (for compression/decompression), ships its own crypto (because the browser crypto APIs are extremely limited and async-only), etc... Ultimately WebAssembly runtimes are very secure, but you have to protect everything else from the code you're running, because the code itself can easily be attacked by uncontrolled input.
- jillesvangurp 3y agoOn paper, this just uses the same security model as javascript and obviously a lot of thought is going into security and sandboxing with this. What has been problematic historically was a lot of native code written in a hurry by dot com era companies being unleashed on browsers via a poorly thought out plugin model. Flash, Silverlight, Java Applets, and loads more stuff existed while people were still OK serving stuff up without SSL, trying to figure out cookies and generally not putting a lot of thought into cross site scripting attacks. That was a security nightmare and all the obvious things happened. WASM does not seem like a repeat of that. Rather it builds on all the learning we've had since then.
- Havoc 3y ago> the ability to bring compute to data eliminates the need for as many microservices — run those in the database instead! Isn’t that a step backwards? De linking them means you can scale the two appropriately independently. I guess if you’re using could and can “steal” a bit of compute that way it makes sense.
- jmull 3y agoI wouldn't call it a step backwards, but if you can put it in the database you shouldn't have been considering microservices in the first place.
- jzoch 3y agoIt depends - usually moving computation closer together is much more performant. You may be underutilizing your DB or spending a lot of roundtrips on networking and this is a good way to reclaim that. It does not prevent you from having services - it just reduces the #.
- CharlesW 3y ago> Isn’t that a step backwards? It could be for some use cases, but there are significant efficiency advantages to processing the data where it lies instead of shipping a copy to separate compute services.
- nilslice 3y ago> De linking them means you can scale the two appropriately independently. It's definitely situational, and this can be true! But, generally its this kind of scaling optimization on everything that has us drowning in microservices today. If we can afford to "steal" some compute from the DB server, and minimize the amount of infra spun up for things that are basically scripts, I consider that a win.
- 10000truths 3y agoIn nearly every practical scenario, the scaling bottleneck lies in network throughput. Trying to scale by putting network I/O between computation and storage is a bit like trying to separate a rocket engine from its fuel tank with a garden hose.
- chrisco255 3y agoDoes anyone have any good resources that cover the implementation of wasm as a plug-in system on either a backend or frotend? Just curious to deep dive on this. Could be a video or in-depth article.
- wikiwong 3y agohave you seen this project? https://extism.org https://extism.org
- zephraph 3y agodylibso, the company who posted this, built extism (mentioned by wikiwong)
- nilslice 3y agoWe sure did =) Thank you, zephraph!
- digikata 3y agoFluvio is an open source data pipeline project [1] w/ plugin wasm modules for data transformations as "SmartModules"[2]. In our case we can run wasm plugins on frontend or backend (none of which require a browser). Feel free to come by our Discord if you have any questions. Disclosure: I work as a engineer for Infinyon backing the project. [1] https://github.com/infinyon/fluvio https://github.com/infinyon/fluvio [2] https://www.fluvio.io/smartmodules/ https://www.fluvio.io/smartmodules/
- chatmasta 3y agoWe (Splitgraph) implemented WASM UDFs in Seafowl [0], a database written in Rust based on Datafusion and optimized for executing queries "at the edge" and returning cache-friendly HTTP responses. Users can call CREATE FUNCTION within an SQL query to create a WASM UDF (docs [1]). We blogged [2] about this feature, and you can read the relevant PRs [3] containing the changes necessary for its implementation. [0] https://seafowl.io https://seafowl.io [1] https://seafowl.io/docs/guides/custom-udf-wasm https://seafowl.io/docs/guides/custom-udf-wasm [2] https://www.splitgraph.com/blog/seafowl-wasm-udfs https://www.splitgraph.com/blog/seafowl-wasm-udfs [3] https://github.com/search?q=repo%3Asplitgraph%2Fseafowl+wasm+author%3Aneumark&type=pullrequests https://github.com/search?q=repo%3Asplitgraph%2Fseafowl+wasm...
- wikiwong 3y agolove seeing this trend of software systems becoming programmable with wasm. exciting times ahead!
- duped 3y agoThe Web Assembly Component Model (1) can't land soon enough. (1) https://github.com/WebAssembly/component-model https://github.com/WebAssembly/component-model
- rapnie 3y agoIs there an indicator on progress on this and related specs (WASI)? It is hard to get a good feel from the repo's.
- duped 3y agoThis is actually a part of WASI, you can follow the WASI preview2 milestone tracker: https://github.com/orgs/bytecodealliance/projects/6 https://github.com/orgs/bytecodealliance/projects/6
- pjmlp 3y agoAgain nothing new on the horizon, DB2, Oracle and MS SQL Server have had JVM and CLR available for UDF for the last 20 years. Naturally one needs to sell WebAssembly as being the first of a kind.
- 10000truths 3y agoWhat WebAssembly brings to the table is the ability for unprivileged users to upload their own arbitrary UDF to the DB for execution.
- paulryanrogers 3y agoThey can't be completely arbitrary without user trust. My guess is security directives like invoker vs creator will need to be specified and checked appropriately whenever the assemblies are installed.
- 10000truths 3y agoThe whole point of having a sandboxed runtime like WebAssembly is that the code can be executed without user trust, as the user cannot pierce the isolation boundary or exceed the resource limits provided by the sandbox.
- paulryanrogers 3y agoHow will that work with RLS and similar controls?
- 10000truths 3y agoI don't see why UDFs and RLS would need to interact, they're orthogonal concepts that can work in tandem. The DB engine filters out the rows that fail the RLS check, then apply the UDF to the remaining rows.
- 3y ago
- s17n 3y agoA bit off topic but as someone who has never used UDFs, how can running them inside of transactions possibly be performant? Is this something which is actually useful for consumer / “web scale” applications or is it mainly just for enterprise apps with small numbers of users?
- eep_social 3y agoDB performance tuning is complicated but well-understood. I don’t see any particular reason that functions run in a db would be any more of a problem than “serverless” in another environment.
- NohatCoder 3y agoDepends on a lot of factors, but most databases have some overhead per call, especially if it happens over a network. If you can write some procedural code that runs on the database and eliminate calls and network traffic that may be a win. Then there is the SQL overhead of parsing and compiling queries, various forms of stored procedures can mitigate that by having a function that is compiled once, and then called with different input data. How this all works varies a lot between databases, so what works in one may not be optimal in another.
- paulddraper 3y agoPlenty of conventional applications run code in a transaction; that's not a novelty. There is a valid concern with using extra CPU/RAM on a (hard to scale) database rather than (easy to scale) application servers. But it all depends on what you are doing, your tenant model, etc.
- jasonthorsness 3y agoBeing able to write UDFs using modern developer tooling, unit test them, lint them, etc. is my favorite thing about WASM UDFs - I’ve always found writing complex SQL UDFs a frustrating experience (disclaimer- I work for SingleStore mentioned in the article and recently spent a lot of time writing C++ to WASM UDFs for that environment).
- sghosh2 3y agoI think what’s interesting is while UDF has existed for some time. It anecdotally has seemed to be increasing in pace in announcements recently driven by wasm. For all the benefits explained in the post leading to a great end user experience, I would expect it’s something all the database providers to roll out as tablestakes in the future.
- dvt 3y agoThe desperation to make WebAssembly a thing is reaching epic proportions. It was always a cutesy little thing that no one took super seriously. A cool experiment that had little real-world application. But non-technical investors poured millions into WebAssembly startups, so we're doomed to keep seeing these posts pop up until all that money dries up. And, trust me, will it ever dry up. Other than a few extremely niche situations (maybe browser-based gaming?) does anyone ever really think to themselves "wow I should really use WebAssembly here?" Even the post itself doesn't talk about any real solutions, just about what might be possible: > database platforms could gain these additions with little to no incremental work required Ultimate cope. It's the classic solution looking for a problem. (Edit: mass downvoted by the WASMOOOORS)
- pphysch 3y ago99% of programmers will never directly touch WASM, but in 10 years 99% of web users will use a WASM-based runtime in their browser for daily tasks/entertainment. WASM will eat SPAs and a chunk of the gaming market, it's just a matter of how long it takes.
- dvt 3y ago> WASM will eat SPAs and a chunk of the gaming market, it's just a matter of how long it takes. Ah yes, the classic "we're still early"—might I introduce you to cryptocurrency? Apologies for the snakiness, but if it hasn't happened in the last 10 years, do you really think it will happen in the next 10?
- malkia 3y agoI remember seeing "gcc/clang/cl -o blah.so/dll/dylib" for an UDF functions (various DBs), and then dlload/loadlibrary/etc it. Nothing new really, but avoids having a compiler, and maybe the `wasm` sandbox helps with security. My only worry in all this (and wasm) in general is how well you can debug this later - be it interactive, or postmortem dump. I'm having my reservations, but would have to experiment and see. So far the best integration I've seen between different languages/runtimes have been C++ / .NET (C#) and C++/CLI (e.g. their managed C++) - I can debug, step in, post-mortem debug without an issue. I can't even do this properly with GoLang, and had mixed success with Python (+.pyd files). Debugging (interactive, and post-mortem) is often overlooked, and thought about later. But I'm all excited about wasm now :)
- nilslice 3y agoI'd encourage you to check out one of our projects, Extism[0], which helps make the kind of integrations you mention way easier! (and using wasm) 0: https://github.com/extism/extism https://github.com/extism/extism
- aklsdjfhaiuehf 3y agoJust wanted to say, this is the best worse idea i've seen in ages! it's on my todo list ;)
- paulddraper 3y agoWebAssembly: achieving the goals of the JVM two decades later.