6 ms·
This reminds me of the Servo project's journey. Always impressed to see another implementation of the WHATWG specs. It's interesting to see Zig being chosen he
by barishnamazov 9mo ago
This reminds me of the Servo project's journey. Always impressed to see another implementation of the WHATWG specs.
It's interesting to see Zig being chosen here over Rust for a browser engine component. Rust has kind of become the default answer for "safe browser components" (e.g., Servo, Firefox's oxidation), primarily because the borrow checker maps so well to the ownership model of a DOM tree in theory. But in practice, DOM nodes often need shared mutable state (parent pointers, child pointers, event listeners), which forces you into Rc<RefCell<T>> hell in Rust.
Zig's manual memory management might actually be more ergonomic for a DOM implementation specifically because you can model the graph relationships more directly without fighting the compiler, provided you have a robust strategy for the arena allocation. Excited to learn from Lightpanda's implementation when it's out.
- fbouvier 9mo agoHi, I am Francis, founder of Lightpanda. We wrote a full article explaining why we choose Zig over Rust or C++, if you are interested: https://lightpanda.io/blog/posts/why-we-built-lightpanda-in-zig https://lightpanda.io/blog/posts/why-we-built-lightpanda-in-... Our goal is to build a headless browser, rather than a general purpose browser like Servo or Chrome. It's already available if you would like to try it: https://lightpanda.io/docs/open-source/installation https://lightpanda.io/docs/open-source/installation
- barishnamazov 9mo agoThanks Francis, appreciate the nice & honest write-up with the thought process (while keeping it brief).
- parhamn 9mo agoOff topic note: I read the website and a few pages of the docs and it's unclear to me for what I can use LightPanda safely. Like say I wanted to swap my it as my engine on playwright, what are the tradeoffs? What things are implemented, what isnt?
- h33t-l4x0r 9mo agoI think it's really more of an alternative to JSDom than it is an alternative to Chromium. It's not going to fool any websites that care about bots into thinking it's a real browser in other words.
- fbouvier 9mo agoThanks for the feedback, we will try to make this clearer on the website. Lightpanda works with Playwright, and we have some docs[1] and examples[2] available. Web APIs and CDP specifications are huge, so this is still a work in progess. Many websites and scripts already work, while others do not, it really depends on the case. For example, on the CDP side, we are currently working on adding an Accessibility tree implentation. [1] https://lightpanda.io/docs/quickstart/build-your-first-extraction-script https://lightpanda.io/docs/quickstart/build-your-first-extra... [2] https://github.com/lightpanda-io/demo/tree/main/playwright https://github.com/lightpanda-io/demo/tree/main/playwright
- epolanski 9mo agoI was actually interested into using lightpanda for E2Es to be honest, because halving the feedback cycle would be very valuable to me.
- infogulch 9mo agoMaybe you should recommend a recipe for configuring playwright with both chromium and lightpanda backends so a given project can compare and evaluate whether lightpanda could work given their existing test cases.
- nicoburns 9mo agoI see you're using html5ever for HTML parsing, and like it's trait/callback based API (me too). It looks like style/layout is not in scope at the moment, but if you're ever looking at adding style/layout capabilities to lightpanda, then you may find it useful to know that Stylo [0] (CSS / style system) and Taffy [1] (box-level layout) are both avaiable with a similar style of API (also Parley [2] which has a slightly different API style but can be combined with Taffy to implement inline/text layout). [0]: https://github.com/servo/stylo https://github.com/servo/stylo [1]: https://github.com/DioxusLabs/taffy https://github.com/DioxusLabs/taffy [2]: https://github.com/linebender/parley https://github.com/linebender/parley --- Also, if you're interested in contributing C bindings for html5ever upstream then let me know / maybe open a github issue.
- quotemstr 9mo agoChoosing something like Zig over C++ on simplicity grounds is going to be a false economy. C++ features exist for a reason. The complexity is in the domain. You can't make a project simpler by using a simplistic language: the complexity asserts itself somehow, somewhere, and if a language can't express the concept you want, you'll end up with circumlocution "patterns" instead. Build system complexity disappears when you set it up too. Meson and such can be as terse as your Curl example. I mean, it's your project, so whatever. Do what you want. But choosing Zig for the stated reasons is like choosing a car for the shape of the cupholders.
- hnlmorg 9mo agoThat’s not fully true though. There’s different types of complexity: - project requirements - requirements forced upon you due to how the business is structured - libraries available for a particular language ecosystem - paradigms / abstractions that a language is optimised for - team experiences Your argument is more akin to saying “all general purpose languages are equal” which I’m sure you’d agree is false. And likewise, complexity can and will manifest itself differently depending on language, problems being solved, and developer preferences for different styles of software development. So yes, C++ complexity exists for a reason (though I’d personally argue that “reason” was due to “design by committee”). But that doesn’t mean that reason is directly applicable to the problems the LightPanda team are concerned about solving.
- vegabook 9mo agoC++ features for complexity management are not ergonomic though, with multiple conflicting ideas from different eras competing with each other. Sometimes demolition and rebuild from foundations is paradoxically simpler.
- Philpax 9mo agoYour Swiss Army Knife with a myriad of 97 oddly-shaped tools may be able to do any job anyone could ask of it, but my Swiss Army Knife of 10 well-designed tools that are optimal for my set of tasks will get my job done with much less frustration.
- meheleventyone 9mo ago
- Jweb_Guru 9mo agoRespectfully, for browser-based work, simplicity is absolutely not a good enough reason to use a memory-unsafe language. Your claim that Zig is in some way safer than Rust for something like this is flat out untrue.
- hello_moto 9mo agoIn that blog post, the author said safer than C not Rust.
- dnautics 9mo agoWhat is your attack model here? Each request lives in its own arena allocator, so there is no way for any potentially malicious JavaScript to escape and read memory owned by any other request, even if there is a miscode. otherwise, VM safety is delegated to the V8 core.
- Jweb_Guru 9mo agoBelieve it or not, using arenas does not provide free memory safety. You need to statically bound allocations to make sure they don't escape the arena (which is exactly how arenas work in Rust, but not Zig). There are also quite a lot of ways of generating memory unsafe code that aren't just use after free or array-out-of-bounds in a language like Zig, especially in the context of stuff like DOM nodes where one frequently needs to swap out pointers between elements of one type and a different type.
- nwienert 9mo agoWould be helpful to compare Lightpanda to Webkit, Playwright has a driver for example and its far faster and less resource hungry than Chrome. When I read your site copy it struck me as either naive to that, or a somewhat misleading comparison, my feedback would be just to address it directly alongside Chrome.
- aatd86 9mo agoWould be great if it could be used as a wasm library... Just saying... Is it? I would actually need and use this.
- pjmlp 9mo agoAnd use-after-free, when that arena's memory goes away.
- pron 9mo agoBut arenas have substantial benefits. They may be one of the few remaining reasons to use a low-level (or "systems programming") language in the first place. Most things are tradeoffs, and the question isn't what you're giving up, but whether you're getting the most for what you're paying.
- pjmlp 9mo agoArenas are also available in languages with automatic memory management, e.g. D, C# and Swift, to use only modern languages as example. Thus I don't consider that a reason good enough for using Zig, while throwing away the safety from modern languages.
- pron 9mo agoFirst, Zig is more modern than any of the languages you mention. Second, I'm not aware that any of those languages offer arenas similar in their power and utility to Zig's while offering UAF-freedom at the same time. Note that "type-safe" arenas are neither as powerful as general purpose arenas nor fully offer UAF-freedom. I could be wrong (and if I am, I'd really love to see an arena that's both general and safe), but I believe that in all these languages you must compromise on either safety or the power of the arena (or both).
- metaltyphoon 9mo ago> First, Zig is more modern than any of the languages you mention How so? This feels like an empty statement at best.
- pron 9mo ago"modern: relating to the present or recent times as opposed to the remote past". I agree it's not a useful concept here but I didn't bring it up. Specifically, I don't think there's any consideration that had gone into the design of D, C#, or Rust that escaped Zig's designer. He just consciously made different choices based on the data available and his own judgment.
- IshKebab 9mo agoI don't think it's really that bad in Rust. If you're happy with an arena in Zig you can do exactly the same thing in Rust. There are a ton of options listed here: https://donsz.nl/blog/arenas/ https://donsz.nl/blog/arenas/ Some of them even prevent use after free (the "ABA mitigation" column).
- mijoharas 9mo agoI'm not super experienced with zig, but I always think that in the same way that rust forces you to think about ownership (by having the borrow checker - note: I think of this as a good thing personally) zig makes you think upfront about your allocation (by making everything that can allocate take an allocator argument.). It makes everything very explicit, and you can always _see_ where your allocations are happening in a way that you can't (as easily, or as obviously - imo) in rust. It seems like something I quite like. I'm looking forward to rust getting an effects system/allocator api to help a little more with that side of things.
- silon42 9mo agoThe problem is deallocation... unless you tie the allocated object to an arena allocator with a lifetime somehow (Rust can model that).
- mijoharas 9mo agoYep, rust forces you to think about lifetimes. Zig only suggests it (because you're forced to think about allocation, which makes you naturally think about the lifetime usually) but does not help you with it/ensure correctness. It's still nice sometimes to ensure that you have to think about allocation everywhere, and can change the allocation strategy for something that works for your usecase. (hence why I'm looking forward to the allocator api in rust to get the best of both worlds).
- IshKebab 9mo agoThat's true and I liked the idea of it until I started writing some Zig where I needed to work with strings. Very painful. I'm sure you typically get a bit faster string manipulation code than what you'd get with Rust but I don't think it's worth the cost (Rust is pretty fast already).
- galangalalgol 9mo agoToo late now, but is the requirement for shared mutable state inherent in the problem space? Or is it just because we still thought OOP was cool when we started on the DOM design?
- pornel 9mo agoYes. It is required for W3C's DOM APIs, which give access to parent nodes and allow all kinds of mutations whenever you want. Event handlers + closures also create potentially complex situations you can't control, and you'll need a cycle-breaking GC to avoid leaking like IE6 did. You can make a more restricted tree if you design your own APIs with immutability/ownership/locking, but that won't work for existing JS codebases.
- 7bit 9mo ago> without fighting the compiler It's unfortunate that "writing safe code" is constantly being phrased in this way. The borrow checker is a deterministic safety net. Claiming Zig is easier ignores that its lack of safety checks is what makes it feel easier; if Zig had Rust’s guarantees, the complexity would be the same. Comparing them like this is apples vs. oranges.
- senko 9mo agoThe fact that Zig doesn't have Rust's guarantees doesn't mean Zig does not have safety checks. The safety checks that Zig does have are different, and are different in a way that's uniquely useful for this particular project. Zig's check absolutely don't go to the extent that Rust's do, which is kind of the point here. If you do need to go beyond safe code in Rust, Zig is safer than unsafe code in Rust. Saying Zig lacks safety checks is unfortunate, although I wouldn't presume you meant it literally and just wanted to highlight the difference.
- pjmlp 9mo agoThing is, those safety checks are also available in C and C++, provided that one uses the right tools like PVS and PurifyPlus (just to quote two examples), and now ongoing AI based tooling efforts for verification, thus the question is why a language like Zig in the 21st century, other than "I don't like either C++ or Rust".
- 7bit 9mo agoI never said Zig has no safety features. What I said is true, though. If it would have Rusts guarantees (as in: The same) it would be more complex.
- senko 9mo agoI mean if we're going to nitpick: >>> its lack of safety checks >> Saying Zig lacks safety checks is unfortunate, > I never said Zig has no safety features. You did. Or, alternatively, if you don't equate "checks" with "features", then I never said you said that so what are you complaining about? > If it would have Rusts guarantees (as in: The same) it would be more complex. Which is true (if tautological), and is basically what the GP said: > Zig's manual memory management might actually be more ergonomic for a DOM implementation specifically because you can model the graph relationships more directly without fighting the compiler, provided you have a robust strategy for the arena allocation Both you and the GP agree that Rust is more complex. You objected to this with: > It's unfortunate that "writing safe code" is constantly being phrased in this way. Upon which I commented that Zig does have safety features, even if they're not covering you as well as Rust's ones. Which is, again, inline with "provided you have a robust strategy for the arena allocation." Now, if you think I'm going overboard with this, I agree with you -- and this is the exact feeling I have when I look at Rust :)
- pron 9mo agoI don't think that a language that was meant to compete with C++ and in 10+ years hasn't captured 10% of C++'s (already diminished) market share could be said to have become "kind of the default" for anything (and certainly not when that requires generalising from n≅1).
- anal_reactor 9mo agoThe problem is that the number of browser engines is n=2.
- drnick1 9mo agoInterestingly, Ladybird, which aims at being the n = 3, is also written in C++.
- drannex 9mo agoLadybird is in the process of switching over to Swift, and has been for a little over a year now. Not linking to the pedophilic nazi-site, and as Nitter is dead-ish, here is the full-text announcement archived on tildes: https://tildes.net/~comp/1j7m/ladybird_chooses_swift_as_its_successor_language_to_c https://tildes.net/~comp/1j7m/ladybird_chooses_swift_as_its_...
- pjmlp 9mo agoIt has for Amazon, Adobe, Microsoft, Google and the Linux kernel. It remains to be seen which big name will make Zig unavoidable.