10 ms·
Wuffs the Language
- dang 6y agoSurprisingly little discussed so far, aside from these past related threads: Wuffs’ PNG image decoder - https://news.ycombinator.com/item?id=26714831 https://news.ycombinator.com/item?id=26714831 - April 2021 (135 comments) C performance mystery: delete unused string constant - https://news.ycombinator.com/item?id=23633583 https://news.ycombinator.com/item?id=23633583 - June 2020 (105 comments) That first one was just yesterday but this is a rare case where we would not downweight the follow-up post (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=follow-up%20by%3Adang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...).
- zokier 6y agoApparently the language was renamed at some point, google/puffs redirects to google/wuffs? Puffs was discussed few years back https://news.ycombinator.com/item?id=15711767 https://news.ycombinator.com/item?id=15711767
- teraflop 6y agoYeah, I was curious about that as well. The README file links to a Google Groups discussion about the name change that seems to have been memory-holed, but apparently it was renamed to avoid confusion with a NetBSD component: https://news.ycombinator.com/item?id=15712659 https://news.ycombinator.com/item?id=15712659
- pjmlp 6y agoThankfully they changed the name, in Germany it would be quite impossible to use it in any public discussion, as Der Puff is a special kind of boys club.
- avmich 6y agoYeah, Coq also feels the heat with name...
- grawprog 6y agoTo be honest, I'm not sure what to make of this. Wuff the library makes sense as a drop in for the C standard library, but the language, I'm not sure how it fits. It seems to offer some of the features offered by languages like D and Rust, while staying more C like, but also removing one of the few actual reasons to use C, which both D and Rust also provide on top of the other features offered by Wuff. It's cool and all but it seems confused as to whether it wants to be a library for C, an extension to C or a standalone language. As a stand alone language, I'm not sure I really see the benefits over alternatives as a C library, it does have some interesting ideas.
- zokier 6y agoThe readme has more explanation: > Wuffs (Wrangling Untrusted File Formats Safely) is formerly known as Puffs (Parsing Untrusted File Formats Safely). > Wuffs is a memory-safe programming language (and a standard library written in that language) for wrangling untrusted file formats safely. Wrangling includes parsing, decoding and encoding. Example file formats include images, audio, video, fonts and compressed archives. > Wuffs is not a general purpose programming language. It is for writing libraries, not programs. The idea isn't to write your whole program in Wuffs, only the parts that are both performance-conscious and security-conscious. For example, while technically possible, it is unlikely that a Wuffs compiler would be worth writing entirely in Wuffs.
- chrchang523 6y agoThe purpose of the language is clear to me: make it practical to prove that >90% of a C file-munging library is safe from several common types of errors. I will be looking into reorganizing some of my existing C library code of this type into large Wuffs components and small interaction-with-outside-world components.
- pnathan 6y agoThis is a fascinating spin: a pure language, designed for libraries, not for complete programs. A tip of the hat to whoever was able to break out of the "a language has to do x y and z" thinking and perceive that this is a possibility.
- tomthe 6y agoYes, an interesting concept. But I guess it won't see a wide adoption: Who wants to learn a new language, if you can't use it for anything but X? And who will use this language for X, if you haven't been able to learn it while doing Y?
- brundolf 6y agoI disagree. It's just a question of how many people are trying to do what it's designed for, and how much benefit it provides for that focused domain.
- enneff 6y agoPeople who want to do X well will learn the language. You can learn more than one language.
- dkersten 6y agoWho would want to learn Javascript for the frontend and Python/Ruby/C#/Java/PHP/Whatever for the backend? And HTML for UI with CSS for styling? You say this, but people do it all the time.
- dkersten 6y agoInterestingly, I was thinking about this exact thing a few hours ago, before I saw this on HN. The thought was that you can design much more interesting languages by not making them be everything for everyone. Not quite domain specific, but also not quite general purpose. I made a language a while back that was used to implement custom logic for a product (I've since replaced it with a more declarative system that's basically TOML but where values can be expressions that get evaluated to generate the actual values). One goal of this language was that it should always terminate[1], so it had no unbounded loops. Another goal was that it should be deterministic, so all input was gathered before execution and all output was accumulated to be processed at the end. The entire thing ran in a database transaction (so input could be queried, then the code was executed as a pure function of this input, then the result would be written back to the database or sent elsewhere). Externally triggered events would cause this to run. Essentially an event driven synchronous[2] language of the transformational system variety. It was slightly inspired by Lustre[3], which is used in critical systems like aeroplanes, trains and power plants. I'm a big fan of this style of language. Basically, by constraining what the language can do or can be used for, you can design much more powerful semantics or language features for the things that it is designed for, similar to a domain specific language. I guess it really would be a somewhat more general domain specific language, or at least domain specific to multiple domains. I was thinking about this while walking home from the shops and was wondering if such a language would be beneficial to solve some challenges I hit in my work and I was going to spend some time thinking what semantics would be useful, but haven't done so yet and then came across this HN submission. :) [1] I was also thinking about how the halting problem doesn't really say that determining if a program halts is impossible, just that there are programs that are not computable. If you add constraints (like not being able to feed the program to itself as is done in the halting problem and not allowing unbounded loops) then it is possible to determine if a program will terminate or not. [2] https://en.wikipedia.org/wiki/Synchronous_programming_language https://en.wikipedia.org/wiki/Synchronous_programming_langua... [3] https://en.wikipedia.org/wiki/Lustre_(programming_language) https://en.wikipedia.org/wiki/Lustre_(programming_language)
- oconnor663 6y ago> There is no operator precedence. A bare a * b + c is an invalid expression. You must explicitly write either (a * b) + c or a * (b + c). Honestly I've often wished for this in mainstream languages. It seems like operator precedence should go the way of bracketless if and implicit int casts. (Though I wonder if they wind up making exceptions here for chains of method calls? I guess technically those rely on operator precedence sort of?) Edit: Yeah I see the example code has "args.src.read_u8?()". So it looks like they figured out how to keep the good stuff.
- wuschel 6y agoLISP-like languages have enforced operator precedence due to polish notation e.g. (+ (* a b) (+ c d))
- sqrt17 6y agothere's no operator precedence if you don't have (multiple) operators that could precede each other. In LISP-like languages these are simply functions (or more correctly, forms) which have other expressions as arguments, like any other functions or forms. LISP works just fine without much of the things we take for granted in ALGOL-like languages.
- aidenn0 6y agoIn addition the variadic prefix-notation means the operators are not limited to being binary: 3*x*y*z+w becomes: (+ (* 3 x y z) w)
- chubot 6y agoWuffs seems fascinating and I really wanted to like it. But when I look at the code for the JSON decoder it seems so low level, and full of places for bugs to hide. JSON is a pretty simple spec and this obscures it (although to be fair it's also handling UTF-8). https://github.com/google/wuffs/blob/main/std/json/decode_json.wuffs https://github.com/google/wuffs/blob/main/std/json/decode_js... Yes it prevents buffer overflows and integer overflow, but it can't prevent logical errors. I'd rather see efficient code generated from a short high level spec, not an overwhelming amount of detail in a language verified along a few dimensions. --- Logical errors in parsing also lead to security vulnerabilities. For example, here is an example of parser differentials in HTTP parsing: https://about.gitlab.com/blog/2020/03/30/how-to-exploit-parser-differentials/ https://about.gitlab.com/blog/2020/03/30/how-to-exploit-pars... The canonical example of this class of bug is forging SSL certificates to take advantage of buggy parsers, but I don't have a link handy. There should be one off of https://langsec.org/ https://langsec.org/ if anyone can help dig it up. Again, this has nothing to do with buffer or integer overflows. (aside: while googling for that I found the claim that mRNA vaccines work by parser differentials: https://twitter.com/maradydd/status/1342891437537505280?lang=en https://twitter.com/maradydd/status/1342891437537505280?lang... If anyone understands that I'd be curious on an opinion/analysis :) ) At the very least, any language for parsing should include support for regular languages (regexes). The RFCs for many network protocols use this metalanguage, and there's no reason it shouldn't be executable. They compile easily to efficient code. The VPRI project claimed to generate a TCP/IP implementation from 200 lines of code, although it's not really a fair comparison because it hasn't been tested in the wild: https://news.ycombinator.com/item?id=846028 https://news.ycombinator.com/item?id=846028 . Still I think that style has better engineering properties. Oil's lexer, which understands essentially all of bash, is generated from a short source file https://www.oilshell.org/release/0.8.8/source-code.wwz/frontend/lexer_def.py https://www.oilshell.org/release/0.8.8/source-code.wwz/front... which generates https://www.oilshell.org/release/0.8.8/source-code.wwz/_devbuild/tmp/osh-lex.re2c.h https://www.oilshell.org/release/0.8.8/source-code.wwz/_devb... which goes on to generate 28,000 lines of C code. It's short, but it really needs a better regex metalanguage to be readable: https://www.oilshell.org/release/latest/doc/eggex.html https://www.oilshell.org/release/latest/doc/eggex.html A large part of JSON can be described by regular languages, and same with HTTP, etc. ----- edit: An re2c target for wuffs could make sense. The generated code already doesn't allocate any memory, although it uses tons of pointers which could be dangling. And in fact that was a problem Cloudflare, which sprayed the user data of their customers all over the Internet back in 2017: https://en.wikipedia.org/wiki/Cloudbleed https://en.wikipedia.org/wiki/Cloudbleed That was with Ragel and not re2c, which perhaps has a more error prone API.
- mjevans 6y agoThe only part of the Wuffs spec I just read that I dislike: Strings. I would really prefer strings to work like existing C and 'bash' style quoting. At least the simple aspects of it, the parts of the rules that are easy to remember and simple. A string should always be a sequence of octets, but easily coerced by a casting operator to a numeric format from any index. I'm not sure what the syntax for that would be offhand.
- rattray 6y agoYeah, when I got to the bottom and saw that they don't have strings, I immediately decided not to spend any more time learning about the language. I'm not sure what I would want them to be like for this language, but I'd definitely want something.
- incrudible 6y agoIt isn't made especially clear on the linked page, but Wuffs is a language to write parsers with. It's not a "general purpose" language, though it might find use in other domains. If you've written a parser, you will have noticed that the built-in string types of other languages are counter-productive. You really want to work with plain ranges over bytes, and Wuffs offers that to you.
- incrudible 6y ago> I would really prefer strings to work like existing C... The way "strings" work in C is a big source of the kind of bugs that Wuffs is supposed to prevent. > A string should always be a sequence of octets, but easily coerced by a casting operator to a numeric format from any index The Wuffs language just uses "a sequence of octets" in favor of a string type.
- lancepioch 6y agoSo a similar sense to Haxe: https://haxe.org https://haxe.org
- chubot 6y agoI think they're pretty different. Haxe is designed to compile to many different languages, sort of like a least common denominator. Whereas Wuffs translates only C, and is pretty semantically close to it. Its goal is really safety, while Haxe seems to be a portability (e.g. running the same game on many platforms).
- peteretep 6y ago> Traditionally, the first program anyone writes in a given programming language is something that prints "Hello world". This doesn't work for Wuffs, for two reasons. One is that Wuffs doesn't have a string type per se. Two is that Wuffs code doesn't even have the capability to write to files directly, such as to stdout. Wuffs is a language for writing libraries, not complete programs, and the less Wuffs can do, the less Wuffs can do that is surprising (such as upload your files to the internet), even when processing untrusted input.
- ledauphin 6y agoI'm confused by the "all functions are methods" restriction. the "what" seems clear, but the "why" is eluding me, and I'd love to read an explanation.
- mkj 6y agoHow does error handling work in Wuffs? That seems to be an important aspect for a reliable language, it wasn't immediately clear from the docs. Edit ah found it, https://github.com/google/wuffs/blob/main/doc/note/statuses.md https://github.com/google/wuffs/blob/main/doc/note/statuses....
- incrudible 6y agoI like the idea. The inability to do something is an often underrated feature.
- peter_d_sherman 6y ago>"Wuffs (Wrangling Untrusted File Formats Safely) is formerly known as Puffs (Parsing Untrusted File Formats Safely). Wuffs is a memory-safe programming language (and a standard library written in that language) for wrangling untrusted file formats safely. Wrangling includes parsing, decoding and encoding. Example file formats include images, audio, video, fonts and compressed archives. It is also fast. On many of its GIF decoding benchmarks, Wuffs measures 2x faster than "giflib" (C), 3x faster than "image/gif" (Go) and 7x faster than "gif" (Rust). Goals and Non-Goals Wuffs' goal is to produce software libraries that are as safe as Go or Rust, roughly speaking, but as fast as C, and that can be used anywhere C libraries are used. This includes very large C/C++ projects, such as popular web browsers and operating systems (using that term to include desktop and mobile user interfaces, not just the kernel). Wuffs the Library is available as transpiled C code. Other C/C++ projects can use that library without requiring the Wuffs the Language toolchain. Those projects can use Wuffs the Library like using any other third party C library. It's just not hand-written C. However, unlike hand-written C, Wuffs the Language is safe with respect to buffer overflows, integer arithmetic overflows and null pointer dereferences. A key difference between Wuffs and other memory-safe languages is that all such checks are done at compile time, not at run time. If it compiles, it is safe, with respect to those three bug classes. The trade-off in aiming for both safety and speed is that Wuffs programs take longer for a programmer to write, as they have to explicitly annotate their programs with proofs of safety. A statement like x += 1 unsurprisingly means to increment the variable x by 1. However, in Wuffs, such a statement is a compile time error unless the compiler can also prove that x is not the maximal value of x's type (e.g. x is not 255 if x is a base.u8), as the increment would otherwise overflow. Similarly, an integer arithmetic expression like x / y is a compile time error unless the compiler can also prove that y is not zero. Wuffs is not a general purpose programming language. It is for writing libraries, not programs. The idea isn't to write your whole program in Wuffs, only the parts that are both performance-conscious and security-conscious. For example, while technically possible, it is unlikely that a Wuffs compiler would be worth writing entirely in Wuffs." PDS: Would like to see a future AV1 / AOM / libaom / FFmpeg -- written/compiled in Wuffs...