6 ms·

This is a really insightful conversation, it's so interesting how much success they have had by "just"* translating the html/css specification directly to code.

This is a really insightful conversation, it's so interesting how much success they have had by "just"* translating the html/css specification directly to code. The "debugging" of the docs that this is achieving is so valuable. Other larger browsers are going to also gain so much from all this insight. Well worth a listen! Slight aside, Andreas has had a bunch of donations and sponsors in the last month (over 300k), and just today announced he has hired a full time dev to help on the browser engin: https://twitter.com/awesomekling/status/1676896595936198657 https://twitter.com/awesomekling/status/1676896595936198657 > Thrilled to announce that I've hired @KalenikW to work full time on @ladybirdbrowser > Alex has been with the project since last year, and already done great work on CSS layout, testing infra, and more. Lately he's digging into JS performance and making great progress! * massive understatement

olliej 3y ago
It’s really important - and people seem to be unwilling to recognize - the huge amount of work done by browser vendors 10-15 years ago to actually make the specs usable for the purpose of implementing a browser (the whatwg work and es3.1 were all about actually building out correct and complete. The w3c specs, and similar era es specs could not be used to implement a browser, and a lot of the browser engineering work of the time was basically trying to find site breakage and then reverse engineering whatever gecko and/or trident did. It meant things that today might take just a few hours to implement by following the spec could take weeks. At one point I spent a month working out what the correct behaviour and sequence of events should be fired when pressing keys on a keyboard because what the specs said both under specified what should happen, and also was just wrong in some edge cases. It was a truly miserable era, and the people who seem hellbent on bashing whatwg and what not for some perceived slight remain absurd to me.
- ahartmetz 3y ago
  That's interesting. The people working on these specs seem to be doing an idealistic work even though their employers' interests are more to keep their moats from having the, more or less, two only "HTML engines" viable on the modern web. Is it really idealism or are there interests at play that I don't know about? Is that stuff too low level to be used as a competitive advantage?
  - zgluck 3y ago
    Around 2004-2007 most of the key web specification writers (From WHATWG; Hixie, etc, etc) a) were employed by Opera Software b) were surprisingly young (often around or even below the age of 18) Then Google poached most of these people (from their partner; Opera had by then invented the placed browser search engine business model for them; allegedly Google is paying Apple $15B/yr these days.) A few years later the scope of the web started growing in an insane manner, spearheaded by Google. This later period was also the same time that the web got deeply javascriptified and largely turned from documents into apps that you download and execute. Before that it had been documents with sprinkles of javascript.
    - olliej 3y ago
      That's some revisionist history. The bulk of the work was people from Mozilla, Apple, and even Microsoft (those that was mostly in the context of TC39/ECMAScript). Opera did contribute - for similar reasons to Apple: the existing specs were abysmal and making usable browsers was hard simply due to the need to actually work out what a browser is meant to do. Some folk like Anne, Hixie, etc were employed by opera at the time, but to portray that as if that is the majority or "key" work is disingenuous. > Then Google poached most of these people As much as I dislike google, claiming you can poach people implies that they're property, and speaks to the anti-poaching lawsuits of a decade or so ago. > A few years later the scope of the web started growing in an insane manner, spearheaded by Google. Google does take a shotgun approach to specifications, and one of the biggest issues when I left (the frustration of dealing with google engineers significantly impacted the burn out I experienced) was them spewing out piles of half assed "specifications" that helped whatever individual project they were working on at the time rather than well thought out specs that provide general solution to the general problem space. The constant battle against that, especially when in a world where forums like HN are filled with people just parroting AlexR style talking points of "Apple is fighting [BS half assed spec] because they hate the open web".
      - zgluck 3y ago
        > speaks to the anti-poaching lawsuits of a decade or so ago. There's a large difference between a) colluding to not hire people between equally sized competitors and b) aggressively reaching out and poaching a (foreign) partner's key people, especially when that partner is 50-100x smaller I'm sure you're aware of this so I'm left a bit confused here. .. > Google does take a shotgun approach to specifications... in a world where forums like HN are filled with people just parroting AlexR style talking points of "Apple is fighting [BS half assed spec] because they hate the open web". Umm... Okay.
- mananaysiempre 3y ago
  The WHATWG approach is not without its downsides, even setting aside the ideological points (e.g. that with no fixed releases you can never be done in any sense): The spec is good and makes you life easy (well, as not-tremendously-difficult as it could ever be) if you go with the LibWeb/Ladybird approach of transcribing it into code line by line; then there is a yawning abyss of performance engineering and conformance bugs a mile wide; then there are all other real implementations, which do not follow the spec word for word for reasons of efficiency. There are a myriad ways to do an LL(1) parser, all well-known to be equivalent; but you are on the hook for proving the validity of every little discrepancy with the WHATWG algorithm that you’ve had to introduce for one reason or another.
  - olliej 3y ago
    Speaking as a person who has implemented numerous web specs, you generally do start with a close approximation to the actual spec - I believe my original Map/Set implementations in JSC were exactly that. Once you have a complete and correct implementation, then you can actually measure things and work out the critical parts and make them perform reasonable well. But it’s really hard to do perf work if you don’t already have a reference/known good implementation. And that’s what LibWeb is doing, which is giving them a workable baseline. Certainly enough that I’ve seen a few Kling videos where he abLe to focus on perf issues. A lot of the core performance bottlenecks devolve to “cache more”, and then devolves to those most hallowed of problems: cache invalidation.
favorited 3y ago
> Andreas has had a bunch of donations and sponsors in the last month I believe that Igalia was the source of $100k of that! Awesome that they, as an open source browser consultancy, can give back to projects like Ladybird.
- ushakov 3y ago
  What would be their incentive to shell out 100k like this? Appreciation?
- rstat1 3y ago
  Of the 3 announced, 2 of them were anonymous and the 3rd was Shopify.
  - favorited 3y ago
    Oops, that's embarrassing. I must have been thinking of sponsorship for Web Engine Hackfest...
DylanSp 3y ago
There's an Eric Lippert post from a few years back about working on the C# compiler, where he followed a similar spec-guided approach, with simpler benefits for both making the code clear and making sure the spec was reasonable: > [...] I wrote all the conversion logic in Roslyn, and I did so for the most part by carefully following the specification. This had two purposes; first, to be sure that I was implementing the actual specification, and second, as yet another chance to ensure that the specification made sense. For much of the conversion code I have methods where each method implements one line of the specification, for this reason. https://ericlippert.com/2015/05/26/a-different-copy-paste-error/ https://ericlippert.com/2015/05/26/a-different-copy-paste-er...
placesalt 3y ago
Wasn't one of the lessons of The Mythical Man Month that building from a spec produced good software? That was talking about System/360, my recollection is that most of the effort went into creating a very detailed and internally consistent specification/manual for the system. Implementation followed the spec, and the results were long-lasting and solid.
- bch 3y ago
  It’s also “conventional wisdom” that (especially?) with the web, there’s The Way It Should Work, and then there’s The Way It Actually Works, which is all the quirks, etc. I’ve not completed an entire server or browser, so not speaking from implementation experience, just the word on the street. Is that stale information now?
  - olliej 3y ago
    The whole whatwg ES3.1/5 saga was dealing with the specs saying what they wanted the world to be (which they did badly and incompletely, and frequently in direct contradiction with) and what browsers needed to do (the “quirks”). The spec as produced by whatwg and the browser vendors actually matched reality, didn’t have ambiguity (or rather any ambiguity that was missed is considered a spec bug/error) and could actually be used to make a browser.
    - bch 3y ago
      That’s heartening to hear. I appreciate you taking the time to point this out.
  - Cyberdog 3y ago
    Mostly stale, yes, thanks in some part to the solidification of the specs that the interview touches on. “Quirks mode” is still a thing to some degree in browser engines, but you’ll get farther than ever adhering to the spec.
    - bch 3y ago

olliej 3y ago

It’s really important - and people seem to be unwilling to recognize - the huge amount of work done by browser vendors 10-15 years ago to actually make the specs usable for the purpose of implementing a browser (the whatwg work and es3.1 were all about actually building out correct and complete. The w3c specs, and similar era es specs could not be used to implement a browser, and a lot of the browser engineering work of the time was basically trying to find site breakage and then reverse engineering whatever gecko and/or trident did. It meant things that today might take just a few hours to implement by following the spec could take weeks. At one point I spent a month working out what the correct behaviour and sequence of events should be fired when pressing keys on a keyboard because what the specs said both under specified what should happen, and also was just wrong in some edge cases. It was a truly miserable era, and the people who seem hellbent on bashing whatwg and what not for some perceived slight remain absurd to me.

ahartmetz 3y ago
That's interesting. The people working on these specs seem to be doing an idealistic work even though their employers' interests are more to keep their moats from having the, more or less, two only "HTML engines" viable on the modern web. Is it really idealism or are there interests at play that I don't know about? Is that stuff too low level to be used as a competitive advantage?
- zgluck 3y ago
  Around 2004-2007 most of the key web specification writers (From WHATWG; Hixie, etc, etc) a) were employed by Opera Software b) were surprisingly young (often around or even below the age of 18) Then Google poached most of these people (from their partner; Opera had by then invented the placed browser search engine business model for them; allegedly Google is paying Apple $15B/yr these days.) A few years later the scope of the web started growing in an insane manner, spearheaded by Google. This later period was also the same time that the web got deeply javascriptified and largely turned from documents into apps that you download and execute. Before that it had been documents with sprinkles of javascript.
  - olliej 3y ago
    That's some revisionist history. The bulk of the work was people from Mozilla, Apple, and even Microsoft (those that was mostly in the context of TC39/ECMAScript). Opera did contribute - for similar reasons to Apple: the existing specs were abysmal and making usable browsers was hard simply due to the need to actually work out what a browser is meant to do. Some folk like Anne, Hixie, etc were employed by opera at the time, but to portray that as if that is the majority or "key" work is disingenuous. > Then Google poached most of these people As much as I dislike google, claiming you can poach people implies that they're property, and speaks to the anti-poaching lawsuits of a decade or so ago. > A few years later the scope of the web started growing in an insane manner, spearheaded by Google. Google does take a shotgun approach to specifications, and one of the biggest issues when I left (the frustration of dealing with google engineers significantly impacted the burn out I experienced) was them spewing out piles of half assed "specifications" that helped whatever individual project they were working on at the time rather than well thought out specs that provide general solution to the general problem space. The constant battle against that, especially when in a world where forums like HN are filled with people just parroting AlexR style talking points of "Apple is fighting [BS half assed spec] because they hate the open web".
    - zgluck 3y ago
      > speaks to the anti-poaching lawsuits of a decade or so ago. There's a large difference between a) colluding to not hire people between equally sized competitors and b) aggressively reaching out and poaching a (foreign) partner's key people, especially when that partner is 50-100x smaller I'm sure you're aware of this so I'm left a bit confused here. .. > Google does take a shotgun approach to specifications... in a world where forums like HN are filled with people just parroting AlexR style talking points of "Apple is fighting [BS half assed spec] because they hate the open web". Umm... Okay.
mananaysiempre 3y ago
The WHATWG approach is not without its downsides, even setting aside the ideological points (e.g. that with no fixed releases you can never be done in any sense): The spec is good and makes you life easy (well, as not-tremendously-difficult as it could ever be) if you go with the LibWeb/Ladybird approach of transcribing it into code line by line; then there is a yawning abyss of performance engineering and conformance bugs a mile wide; then there are all other real implementations, which do not follow the spec word for word for reasons of efficiency. There are a myriad ways to do an LL(1) parser, all well-known to be equivalent; but you are on the hook for proving the validity of every little discrepancy with the WHATWG algorithm that you’ve had to introduce for one reason or another.
- olliej 3y ago
  Speaking as a person who has implemented numerous web specs, you generally do start with a close approximation to the actual spec - I believe my original Map/Set implementations in JSC were exactly that. Once you have a complete and correct implementation, then you can actually measure things and work out the critical parts and make them perform reasonable well. But it’s really hard to do perf work if you don’t already have a reference/known good implementation. And that’s what LibWeb is doing, which is giving them a workable baseline. Certainly enough that I’ve seen a few Kling videos where he abLe to focus on perf issues. A lot of the core performance bottlenecks devolve to “cache more”, and then devolves to those most hallowed of problems: cache invalidation.

favorited 3y ago

> Andreas has had a bunch of donations and sponsors in the last month I believe that Igalia was the source of $100k of that! Awesome that they, as an open source browser consultancy, can give back to projects like Ladybird.

ushakov 3y ago
What would be their incentive to shell out 100k like this? Appreciation?
rstat1 3y ago
Of the 3 announced, 2 of them were anonymous and the 3rd was Shopify.
- favorited 3y ago
  Oops, that's embarrassing. I must have been thinking of sponsorship for Web Engine Hackfest...

DylanSp 3y ago

There's an Eric Lippert post from a few years back about working on the C# compiler, where he followed a similar spec-guided approach, with simpler benefits for both making the code clear and making sure the spec was reasonable: > [...] I wrote all the conversion logic in Roslyn, and I did so for the most part by carefully following the specification. This had two purposes; first, to be sure that I was implementing the actual specification, and second, as yet another chance to ensure that the specification made sense. For much of the conversion code I have methods where each method implements one line of the specification, for this reason. https://ericlippert.com/2015/05/26/a-different-copy-paste-error/ https://ericlippert.com/2015/05/26/a-different-copy-paste-er...

placesalt 3y ago

Wasn't one of the lessons of The Mythical Man Month that building from a spec produced good software? That was talking about System/360, my recollection is that most of the effort went into creating a very detailed and internally consistent specification/manual for the system. Implementation followed the spec, and the results were long-lasting and solid.

bch 3y ago
It’s also “conventional wisdom” that (especially?) with the web, there’s The Way It Should Work, and then there’s The Way It Actually Works, which is all the quirks, etc. I’ve not completed an entire server or browser, so not speaking from implementation experience, just the word on the street. Is that stale information now?
- olliej 3y ago
  The whole whatwg ES3.1/5 saga was dealing with the specs saying what they wanted the world to be (which they did badly and incompletely, and frequently in direct contradiction with) and what browsers needed to do (the “quirks”). The spec as produced by whatwg and the browser vendors actually matched reality, didn’t have ambiguity (or rather any ambiguity that was missed is considered a spec bug/error) and could actually be used to make a browser.
  - bch 3y ago
    That’s heartening to hear. I appreciate you taking the time to point this out.
- Cyberdog 3y ago
  Mostly stale, yes, thanks in some part to the solidification of the specs that the interview touches on. “Quirks mode” is still a thing to some degree in browser engines, but you’ll get farther than ever adhering to the spec.
  - bch 3y ago