4 ms·
This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist
by gavinray 4mo ago
This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist:
> "Each self-describing F3 file includes both the data and meta-data, as well as WebAssembly (Wasm) binaries to decode the data. Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native decoders are unavailable. "
- verdverm 4mo agoIs embedding executable code into a file a security risk? My assumption is a yes
- mirashii 4mo agoThat would be why it chose a VM that is explicitly designed for sandboxing rather than native executable code or similar, the risk can be minimized by reducing the surface area available to that executable code to almost nothing.
- jayd16 4mo agoYou still have the halting problem to solve to prevent denial of service.
- adwn 4mo agoSolving the halting problem is neither necessary nor sufficient to prevent DoS attacks. It isn't necessary, because settings timeouts or other resource restrictions works way better to prevent DoS. It isn't sufficient, because even if you can prove that a program will halt at some point, this alone doesn't tell you how long it will take. What good does it do to know that the program will run for 10 years before it halts? By that time, service will already have been denied. Even turning hash table lookups from O(1) to O(n) (still very much terminating!) can result in a DoS.
- jayd16 4mo agoIt was just a joke, bud. I agree that a timeout would likely be fine depending on the situation. This is basically just a web page with javascript.
- gavinray 4mo agoThere is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.
- outside1234 4mo agoI mean can't we say the same thing about sending around a .exe though?
- jastanton 4mo agoexactly
- gavinray 4mo agoDouble-clicking an ".exe" (or running it via a shell) is not the same as "bag of bytes", it's "send these bytes to an executable environment". Doing `head foo.exe` is quite different than `run foo.exe` If I encode executable instructions in "image.png" and then send them to an interpreter that runs those instructions, the file extension doesn't matter.
- bluejekyll 4mo ago.exe has bindings to OS ABI and system calls, WASM doesn’t have this by default, it’s up to the VM to provide whatever environment the WASM executable needs, ideally there should be no system calls, no stdio, just instructions on how to interpret the file format.
- jastanton 4mo agogotcha, so the vulnerability will be in some common libraries that attackers force some wasm fallback path with custom wasm instructions that when executed does something nefarious. I'd say at worst it's setup for poor security
- jedberg 4mo agoSure, but when the standard says "read this file and execute the instructions you find at the beginning" that is more dangerous than "this is a file with data and your program needs to figure out how to read it".
- msla 4mo ago> Is embedding executable code into a file a security risk? Yes, which is why nobody uses PDFs.
- NooneAtAll3 4mo agowhich is why no sane pdf viewer implements executable features*
- bguebert 4mo agoI mean I disable javascript embedded in pdf and feel like it would have been better to not have that feature. It would spare people from the invoice.pdf email attachment viruses because most people had assumed pdf isn't going to be as bad as an exe.
- nine_k 4mo agoTrueType and OpenType fonts include code executed by a VM to even render them. This wasn't a viable source of attacks so far, due to the properly limited nature of the VMs. Maybe I would pick the eBPF VM instead, with all its limiting and verifying mechanics.
- tedd4u 4mo agoThere are many documented, exploited-in-the-wild font-file attacks (one example in 1]). Apple is re-writing their font interpreter specifically to improve security. [2] [1] https://www.bleepingcomputer.com/news/security/facebook-discloses-freetype-2-flaw-exploited-in-attacks/ https://www.bleepingcomputer.com/news/security/facebook-disc... [2] https://blakecrosley.com/blog/truetype-hinting-swift-migration https://blakecrosley.com/blog/truetype-hinting-swift-migrati...
- cmiles74 4mo agohttps://learn.microsoft.com/en-us/security-updates/SecurityBulletins/2011/ms11-087?redirectedfrom=MSDN https://learn.microsoft.com/en-us/security-updates/SecurityB... > This security update resolves a publicly disclosed vulnerability in Microsoft Windows. The vulnerability could allow remote code execution if a user opens a specially crafted document or visits a malicious Web page that embeds TrueType font files. > This security update is rated Critical for all supported releases of Microsoft Windows. For more information, see the subsection, Affected and Non-Affected Software, in this section. > The security update addresses the vulnerability by modifying the way that a Windows kernel-mode driver handles TrueType font files. For more information about the vulnerability, see the Frequently Asked Questions (FAQ) subsection for the specific vulnerability entry under the next section, Vulnerability Information.
- jasonjayr 4mo agoSo attackers don't have to craft specially corrupted files? They can just include the code to perform the attack in the data file itself?
- arcfour 4mo agoYes...my first thought. No way in hell anyone actually trusts this. (And as if we didn't trust the compiler enough already!)
- doctorpangloss 4mo agoBut the WASM runs in the sandbox! It only has access to some files, your display, inputs, ... nothing insecure at all!
- gavinray 4mo agoWASM runs in a confined memory space allocated for the program. There is no I/O or host address space access. You need to run a WASI environment for that.
- nine_k 4mo agoDoes WASM have built-in I/O? If not, all that a decoder would be able to do is to decode into a buffer.
- 0x457 4mo ago
- grodes 4mo agoHow is wasm better than C bindings?
- gavinray 4mo agoMany languages don't have ergonomic experiences for working with C ABI's without explicit wrapper code. Hell, Node.js didn't even get this ability until LAST MONTH: https://nodejs.org/en/blog/release/v26.1.0 https://nodejs.org/en/blog/release/v26.1.0 You'd have to write a second library to interface the C ABI with Node via NAPI just to consume it.
- bluejekyll 4mo agoWASM is platform independent. What do you mean by C bindings? C bindings to what?
- grodes 4mo agoC bindings to a C implementation
- yung_lean 4mo agoThis isn't using WASM to solve the "how can I make my file format compatible with more programming languages?" problem. This is trying to solve the "how can I add new encodings to my file format without making everyone update their code?" problem. The former would rightly be solved with C bindings that anyone can link with if they want. The latter might not seem like a big deal, but it's been the main blocker advancing the parquet format. Most people end up not caring about new advanced encodings and just write parquet files with the most compatible feature set.
- coldtea 4mo agoC bindings are not platform independent, nor do they come with a runtime and a sandbox, among other things. Apples to oranges.
- andrewstuart2 4mo agoI would call it clever. I'm not sure I'd call it genius. When I'm working with data I'm working in a specific set of languages. Usually one. Yeah, other people might be working in other languages, but no individual author really needs a language-agnostic way of accessing data beyond compile time. Add to that the likely runtime boundaries that may need to be crossed instead of e.g. inlined by the compiler because it's in-language and dealing with known offsets or tags (depends on the data format of course). To the other commenter's point, am I going to have to sandbox all data access code just to be sure it's not able to do something unexpected? There's a lot of complexity here. And the inherent risk is going to slow down the operation that should be the simplest and fastest: interpreting bytes.
- yung_lean 4mo agoA big problem with parquet, which this aims to replace, is that it's hard to add new encodings because everyone wants to stay compatible with old readers. Embedding the decoders in the file as WASM solves this problem since in theory, old readers will be able to read new files by just using the provided WASM to decode a column whose format the reader doesn't recognize. So this is really about making a file that is forwards compatible in a way that lets you push the standards more than existing formats.
- coldtea 4mo ago>no individual author really needs a language-agnostic way of accessing data beyond compile time. That's so untrue! People need language-agnostic ways to access data all the time, and people work with data accessing them from multiple languages all the time! If I have parquet files I can load them in duckdb, in pandas and polars, process them with various independent tools, and loads of other things... and people do that. This is also why people like something like an SQL database, your data is not locked to some specific language / lib for access.
- rebeccajae 4mo agoIt sounds neat, but feels like it might fall apart with higher-complexity formats. What does an embedded decoder for a PDF look like? I guess since they are tightly-coupled to the file bytes themselves, the author of the file gets to choose what formats make sense, but not all formats have a one-true-decode-step.
- aseipp 4mo agoDespite the name seemingly implying otherwise, F3 is an alternative to columnar storage formats like Parquet; the goal is not to support every conceivable encoding of every file type such as a PDF. Think of the use cases being more like "What if you used a specialized compressor and need a custom block decompression algorithm" or "Decode internal format into Arrow output" or something like that.
- cbm-vic-20 4mo agoApplets redux.
- mort96 4mo agoI don't understand how that's supposed to work. What does the decoder decode into? That's gonna depend entirely on the kind of data, right? For some formats, it's gonna be a stream of bytes; for others, a 2D plane of pixels; others again will need vertexes, 2D planes of pixels and UV maps; for some, an object graph will make more sense.
- gavinray 4mo agoIt appears as though the WASM decode returns two values -- one indicating the data type as a primitive value, and a second value being the data buffer Then there is a helper in this case to de-serialize, "primitive_array_from_buffers()" https://github.com/future-file-format/F3/blob/bd92506447dc1333820cfe9a34ebbf497fe0219e/fff-ude-wasm/examples/wasm.rs#L137-L157 https://github.com/future-file-format/F3/blob/bd92506447dc13...
- vouwfietsman 4mo agoexcept you need flatbuffers to access that blob