9 ms·
Chat with Andreas Kling about Ladybird and developing a browser engine
- samwillis 3y agoThis is a really insightful conversation, it's so interesting how much success they have had by "just"* translating the html/css specification directly to code. The "debugging" of the docs that this is achieving is so valuable. Other larger browsers are going to also gain so much from all this insight. Well worth a listen! Slight aside, Andreas has had a bunch of donations and sponsors in the last month (over 300k), and just today announced he has hired a full time dev to help on the browser engin: https://twitter.com/awesomekling/status/1676896595936198657 https://twitter.com/awesomekling/status/1676896595936198657 > Thrilled to announce that I've hired @KalenikW to work full time on @ladybirdbrowser > Alex has been with the project since last year, and already done great work on CSS layout, testing infra, and more. Lately he's digging into JS performance and making great progress! * massive understatement
- olliej 3y agoIt’s really important - and people seem to be unwilling to recognize - the huge amount of work done by browser vendors 10-15 years ago to actually make the specs usable for the purpose of implementing a browser (the whatwg work and es3.1 were all about actually building out correct and complete. The w3c specs, and similar era es specs could not be used to implement a browser, and a lot of the browser engineering work of the time was basically trying to find site breakage and then reverse engineering whatever gecko and/or trident did. It meant things that today might take just a few hours to implement by following the spec could take weeks. At one point I spent a month working out what the correct behaviour and sequence of events should be fired when pressing keys on a keyboard because what the specs said both under specified what should happen, and also was just wrong in some edge cases. It was a truly miserable era, and the people who seem hellbent on bashing whatwg and what not for some perceived slight remain absurd to me.
- ahartmetz 3y agoThat's interesting. The people working on these specs seem to be doing an idealistic work even though their employers' interests are more to keep their moats from having the, more or less, two only "HTML engines" viable on the modern web. Is it really idealism or are there interests at play that I don't know about? Is that stuff too low level to be used as a competitive advantage?
- zgluck 3y agoAround 2004-2007 most of the key web specification writers (From WHATWG; Hixie, etc, etc) a) were employed by Opera Software b) were surprisingly young (often around or even below the age of 18) Then Google poached most of these people (from their partner; Opera had by then invented the placed browser search engine business model for them; allegedly Google is paying Apple $15B/yr these days.) A few years later the scope of the web started growing in an insane manner, spearheaded by Google. This later period was also the same time that the web got deeply javascriptified and largely turned from documents into apps that you download and execute. Before that it had been documents with sprinkles of javascript.
- olliej 3y agoThat's some revisionist history. The bulk of the work was people from Mozilla, Apple, and even Microsoft (those that was mostly in the context of TC39/ECMAScript). Opera did contribute - for similar reasons to Apple: the existing specs were abysmal and making usable browsers was hard simply due to the need to actually work out what a browser is meant to do. Some folk like Anne, Hixie, etc were employed by opera at the time, but to portray that as if that is the majority or "key" work is disingenuous. > Then Google poached most of these people As much as I dislike google, claiming you can poach people implies that they're property, and speaks to the anti-poaching lawsuits of a decade or so ago. > A few years later the scope of the web started growing in an insane manner, spearheaded by Google. Google does take a shotgun approach to specifications, and one of the biggest issues when I left (the frustration of dealing with google engineers significantly impacted the burn out I experienced) was them spewing out piles of half assed "specifications" that helped whatever individual project they were working on at the time rather than well thought out specs that provide general solution to the general problem space. The constant battle against that, especially when in a world where forums like HN are filled with people just parroting AlexR style talking points of "Apple is fighting [BS half assed spec] because they hate the open web".
- favorited 3y ago> Andreas has had a bunch of donations and sponsors in the last month I believe that Igalia was the source of $100k of that! Awesome that they, as an open source browser consultancy, can give back to projects like Ladybird.
- DylanSp 3y agoThere's an Eric Lippert post from a few years back about working on the C# compiler, where he followed a similar spec-guided approach, with simpler benefits for both making the code clear and making sure the spec was reasonable: > [...] I wrote all the conversion logic in Roslyn, and I did so for the most part by carefully following the specification. This had two purposes; first, to be sure that I was implementing the actual specification, and second, as yet another chance to ensure that the specification made sense. For much of the conversion code I have methods where each method implements one line of the specification, for this reason. https://ericlippert.com/2015/05/26/a-different-copy-paste-error/ https://ericlippert.com/2015/05/26/a-different-copy-paste-er...
- placesalt 3y agoWasn't one of the lessons of The Mythical Man Month that building from a spec produced good software? That was talking about System/360, my recollection is that most of the effort went into creating a very detailed and internally consistent specification/manual for the system. Implementation followed the spec, and the results were long-lasting and solid.
- bch 3y agoIt’s also “conventional wisdom” that (especially?) with the web, there’s The Way It Should Work, and then there’s The Way It Actually Works, which is all the quirks, etc. I’ve not completed an entire server or browser, so not speaking from implementation experience, just the word on the street. Is that stale information now?
- olliej 3y agoThe whole whatwg ES3.1/5 saga was dealing with the specs saying what they wanted the world to be (which they did badly and incompletely, and frequently in direct contradiction with) and what browsers needed to do (the “quirks”). The spec as produced by whatwg and the browser vendors actually matched reality, didn’t have ambiguity (or rather any ambiguity that was missed is considered a spec bug/error) and could actually be used to make a browser.
- bch 3y agoThat’s heartening to hear. I appreciate you taking the time to point this out.
- Cyberdog 3y agoMostly stale, yes, thanks in some part to the solidification of the specs that the interview touches on. “Quirks mode” is still a thing to some degree in browser engines, but you’ll get farther than ever adhering to the spec.
- bch 3y ago
- zgluck 3y ago> Google throws hundreds of engineers at this [building Chrome] and they have full-time people working on it around the clock. It was many hundreds already about a decade ago. I haven't done any recent research (the Chromium Git repo is where you'd get the source data to analyze), but I assume it's in the (several?) thousands now. Edit (ran into some problem replying to the comment below): An in-depth analysis performed today would need to consider that some of the active chromium.org committers are working on non-browser-related things like Chrome OS. That said, I'm assuming 75+% of active chromium.org committers don't work on those non-browser parts. I also consider the Chrome Web Store a part of the browser - but I can't imagine that needing lots of developers in comparison to these large numbers, even when you start thinking about the abuse aspects. Having said that, it should be easy to filter out those by just discarding certain directories.
- kevingadd 3y agoThe way lots of things are integrated into "Chrome" as an organization and a product definitely means a lot more people touch something you can call "Chrome" now. Chrome OS, Chrome Web Store, etc.
- mike_hearn 3y agoLast time I looked the 7 day active committers was about 750. So with holidays, people who don't commit for a week, PMs and other management etc, my guess is around ~1000 people, maybe a bit more. I doubt it's several thousand. Maybe if you include ChromeOS?
- zgluck 3y agoThanks! That is still a lot to compete with, but it's not that much larger than what they had a decade ago. And loads of them were certainly working on mostly homegrown web standards that went nowhere. I think Kling and his growing crew has a shot here.
- refulgentis 3y ago2?
- amadeuspagel 3y ago[flagged]
- pipeline_peak 3y ago[flagged]
- dizhn 3y agoWhat happened here? I don't get either of these two comments.
- pipeline_peak 3y agoOP pointed out the irony in saying there’s no better time to build a browser when the spec documentation is more advanced. I took that as it’s more difficult because the specs are more advanced. Andreas might’ve meant there’s more information provided by browser documentation. If that’s the case, I disagree because Google practically leads by their own standard through usershare. so they’re free to move the goal post.
- seti0Cha 3y agoI think Andreas' point is that although there are more specs and they contain more details, they are far less ambiguous so that a careful reading allows one to achieve correct functionality without guesswork. Since he's worked on browsers for many years, I'm inclined to believe him. Also, if you watch his browser hacking videos, you can actually see this play out live. He finds a layout problem, searches for the relevant spec, and reads through until he finds which detail was missed. He then literally copies that spec chunk into the code, then conforms the code to it. I find it fascinating to watch and I think justifies his claim. As for Google moving the goal posts, if you read through the interview, you can see he's not interested in following bleeding edge, he's interested in the solidified parts of the spec that people actually use today.
- foobarbaz123456 3y ago
- Vinnl 3y agoI'm looking forward to the Servo-Ladybird browser wars.
- placesalt 3y ago(Irrelevant post, but this was a missed opportunity for an article with the title "Kling on Ladybird")
- deleted 3y ago[deleted]
- andrewstuart 3y agoChrome and Firefox are absolutely gargantuan in terms of their functionality and also the decades of fine tuning. Building a new browser seems akin to building a Saturn V rocket. To be practical at all you’d have to lop gigantic trunks of functionality off it.
- niutech 3y agoIf by web browser you mean web engine + chrome, than it is a big task, but notas big as Saturn V. Have a look at Netsurf, Ekioh Flow. Their teams are small, yet still capable.
- samwillis 3y agoI'm not sure I agree with this metaphor but it's too tempting not to propose: Labybird/LibWeb is to Chrome/Blink as Saturn V is to SpaceX Starship... Not sure where that puts Andreas, hence my hesitation.
- KRAKRISMOTT 3y agoMaybe you mean the SLS (or Rocket Lab) instead of Saturn V
- jklinger410 3y agoAn observation shared by many.
- MatthiasPortzel 3y agoJust yesterday I was thinking about Ladybird, and I claimed that it was a fool’s errand, for reasons I’ll explain in a bit. What I didn’t know is if Kling recognized what he was doing, and was just not taking it very seriously, or if he was way over his head and was kind of oblivious. I enjoyed reading this interview because it makes it very clear that Kling has a lot of experience doing this, and understands that’s it’s difficult, and is doing it because he enjoys it. The reason I don’t think Ladybird will ever really get to the point where it can render an arbitrary site perfectly, is based on my experience as a web developer. I haven’t been doing web development for very long, but there have been a couple times in the last couple of years where I’ve stumbled upon browser layout bugs where Chrome/Firefox/Safari render something different. These are normally not issues where a browser completely misses something (ie, every browser supports flexbox these days). Rather, there are edge-case bugs that appear just by combining simple web components in simple ways. Chrome and Firefox handle a table with percentage height differently. Or a margin on a child which is larger than the parent’s width. And these are bugs that have existed for years. And for every edge-case like that where Chrome and Firefox differ, there are 10 more where the behavior is unintuitive or not defined by the spec but Firefox and Chrome have standardized. But maybe, since Ladybird doesn’t have an existing buggy implementation, they’ll be able to implement the spec properly quickly the first time. I think that’s an interesting perspective that Kling brings up in this interview. The other thing I want to comment on is that it’s really hard to tell how much progress they’re making. > We've taken a very vertical slice approach to adding functionality to the browser… We tend to find a webpage that we want to improve for whatever reason. Maybe it's cool to get the New York Times to render, and then we just spend a bunch of time fixing as many issues as we can for that site… And that might not be the most structured approach to it, but it has been very, very good at keeping people motivated and excited and engaged in the work This means that Ladybird “supports” a ton of features. They support flexbox! They support grid! But pretty much all of these features are only partial implementations. Kling says in the interview they support svg, and then later explains that they’re going to have to re-write the text-rendering system to get text-on-curves working. Well, to me, “supports svg” means “we have implemented the entire svg specification.” When Kling is using it to mean “we have some portion of svg’s work. We don’t crash when we see an svg element.” I’m not saying Kling’s definition is wrong, but it’s another factor that makes it difficult to judge where the project is. To start to wrap up this (long) comment, let me restate the four claims I’m making. (This is not the conclusion, this is the summary before the conclusion, hang on.) * Ladybird is developed differently in two ways: vertical slices, and being implemented from scratch in modern times. * there are people who are misled by the vertical-slices and Kling’s use of “support” to believe that Ladybird development is a lot farther along than it is. * There are people like me who think that Ladybird will never complete with existing browser engines (we also have a hard time judging Ladybird’s progress for the reasons stated above). * These same reasons may allow Ladybird to reach compatibility with other browsers. I have a lot of respect for Kling, and I think it’s cool that they’ve been able to accomplish so much “for the fun of it”. As mentioned in the interview, this will have short-term positive impact even if it never matches Chrome. If it wasn’t clear from how long this was, I’m actually really enthusiastic about the project, and I’ll be looking at ways to contribute in the future.
- bilekas 3y agoToday i learn you need to explain a reason to the NYT before you can cancel a subscription! Incredible!
- Demolotoooo 3y agoI will still not try it out. There is too much security issues by just browsing the web. Security is hard and thanks to browser we are moving a lot into quite secure sandbox.
- ilrwbwrkhv 3y agoAndreas Kling is a true hacker, one of the remaining few. A beacon of the hacker spirit. May he sail long in the winds of the incessant specs and guide us. I hope he inspires thousands more to eschew LeetCode and be a true hacker.