6 ms·

When a measure becomes a target, it ceases to be a good measure. Which is to ask: So they improved their Lighthouse score, grats, but did they improve or reduc

When a measure becomes a target, it ceases to be a good measure. Which is to ask: So they improved their Lighthouse score, grats, but did they improve or reduce their user experience? Where's the data on that? This is of course caused by Google's "good intentions" gone awry. They originally created Lighthouse to help better inform developers, so that developers could create better user experiences. But Google is now [ab]using Lighthouse as a target for Google Search page-position preference. Developers in turn given this situation code towards Lighthouse (even into its bugs/quirks) because they need that higher ranking, even if it hurts the UX ultimately.

vlucas 5y ago
I agree with the premise here, but we literally added ~100px to the width of our really small thumbnail images that were hard to see with most modern screen resolutions. People come to the website to view property photos. They were literally smaller than a 256x256 map tile. The size of them should have been increased long ago.
- vanderZwan 5y ago
  Well, I'm glad that worked out in your case, but that's just getting lucky, no? Sounds like in most other scenarios it would backfire.
  - dylan604 5y ago
    That's not the take I got from reading the write up. There was logic in the decision to increase the size that was not difficult to understand. I'm not familiar with what Lighthouse is/does/why/etc, but they described the situation enough for this to make total sense on why it worked. Maybe they are lucky to have people working for them that can work the problem?? That's me stretching to accept lucky in the least as the reasoning.
    - mediumdeviation 5y ago
      It only worked because Google Maps loads images in 250px tiles. If Google Maps decided to update tile size tomorrow, the fix regresses. Arguably the entire map as a whole is one content, and that should be measured as the LCP, but more importantly it shows the measure is just very brittle and easily gamed, as other comments in this thread already attest to.
      - dylan604 5y ago
        Does setting the size inline with width="250" in the embed affect the LCP even if G decides to default to 512px? I'm actually surprised there's not a ?width=250 value in the G embed url
    - Aeolun 5y ago
      > There was logic in the decision to increase the size that was not difficult to understand. I agree, but the logic doesn’t make sense if your goal is ‘optimizing the user experience’ as opposed to ‘optimizing our lighthouse score’.
    - vanderZwan 5y ago
      Apologies, I wasn't specific enough in expressing what I meant. You understood it to mean "they didn't know what they were doing and got lucky", which is a valid interpretation but not what I was going for. They definitely do know what they're doing. What I meant was more along the lines of "the metric to optimize for was no longer strongly correlated to the original goal of improving the user experience. They were lucky that in this scenario optimizing for the metric aligned with an improved user experience."
      - dylan604 5y ago
        Now here's me doing some back bending playing devil's advocate. If the image being optimized for UX resulted in the wacky scoring situation that lowered their ranking in search, would that also not be considered a bad UX if it's harder for users to find you? The team noticed the lower rankings, then investigated. They then realized why the ranking was lower due to the bad LCP. The investigation found that their image optimization was causing the bad LCP, so a quick change on that fixed the bad LCP. So it's not like they were chasing a metric for metric's sake. They did the correct optimization first. So again, I think calling it lucky is a bit harsh in this case. However, I totaly get where you are coming from and know that a lot of places will luck there way into things without ever knowing the why.
rikroots 5y ago
I was surprised that when I tested my canvas library website using the new Lighthouse metrics it scored 92 on performance (and 99 on accessibility, which is the metric I most care about). The website[1] includes a massive animated banner in a <canvas> element which requires loading two large (yet responsive) images alongside processing etc to get the animation playing. The LCP measure was 0.7s Checking web.dev's LCP page[2] explains my luck - Lighthouse does not (currently) consider <canvas> elements, or their asset requirements, when calculating the LCP score. They do include: "<img> elements; <image> elements inside an <svg> element; <video> elements (the poster image is used); an element with a background image loaded via the url() function (as opposed to a CSS gradient); and block-level elements containing text nodes or other inline-level text elements children." I am certainly not suggesting that everyone starts loading their image-heavy banner content into <canvas> elements to help improve their LCP scores as I fully expect Google to address such attempts to game the system at some point in the (near) future. [1] - https://scrawl-v8.rikweb.org.uk/ https://scrawl-v8.rikweb.org.uk/ [2] - https://web.dev/lcp/ https://web.dev/lcp/
jchw 5y ago
Well, they went with a simpler approach earlier, with AMP. AMP couldn’t make any literal guarantees about UX or performance, but in comparison to the mobile experiences reading articles before it, it did seem to often be an improvement. And of course, people hated it, for numerous reasons. You could just make pages faster and less annoying by using plain HTML, obviously. The problem isn’t that anyone knew this, it’s that most of the big publications DGAF, and they were the entire point of AMP as far as I can see. There are many reasons to dislike the way AMP was executed, even if they did eventually rectify many of the serious issues... but the AMP cache concept in combination with the soured reputation probably spelled the end of it no matter what would happen next. AMP for E-mail doesn’t really have any of those problems, so I suspect AMP will probably live on in e-mail. But Google does definitely want to push sites towards better performance and users towards better performing sites; they simply have a mutual interest in this, I don’t think most people would dispute that. Even just in general, but especially when new Android phones are not packing the same punch as new iPhones are... so that scroll jank is going to hit a lot harder. Lighthouse is generally good. It actually discourages a lot of things that people already thought were bad ideas, even ideas from Google. It tends to score plain HTML pages very well, and if it does catch a snag on something you can usually at least work around it. It provides decent diagnostics for how to optimize a web page. It does have some issues, and that will contort people into doing otherwise silly things. But if the website in question chooses a metric in something like Lighthouse in a way that is detrimental to their UX, chances are they just never gave a shit about UX to begin with. While I don’t want to come off as a shill, and I also don’t want to come off as accepting Lighthouse just because it’s “less bad” than AMP, I just think that the “abuse” here is less substantiated. Although there are edge cases where Lighthouse can be silly, it does generally force you to make leaner pages. I think it makes a decent measure of performance. As long as a few points on Lighthouse is not make-or-break for SEO scores, and I doubt it’s that sensitive given the variability per-run, I think this is actually a good thing. Search engine metrics are always imperfect. Anyone familiar with Wikia/Fandom can understand how its subpar content can overshadow much better community-driven websites due to its SEO position, and that is even in spite of what an awful, slow, ad-ridden pile of shit it is. I’d be delighted to see it Page 2’d over Lighthouse scores. But that’s just my feelings on the matter. I’m sure there’s more to it than that.
kmeisthax 5y ago
In this particular case the developer had wondered if chasing Lighthouse was a good idea, but realized that they had already made a similar change on another site for different reasons. Better Lighthouse scores were just a bonus.
linkjuice4all 5y ago
It's been annoying seeing these metrics creep into marketing reports and seeing devs wasting hours "fixing" the issues. Page speed is definitely an issue but I rarely see these stories accompanied by actual user speed data from the site. Google Analytics captures this stuff so you can actually compare pages per session, time on page, scroll distance, etc to paint, load, and render times to "quantify" what users are actually seeing when they use your product.

vlucas 5y ago

I agree with the premise here, but we literally added ~100px to the width of our really small thumbnail images that were hard to see with most modern screen resolutions. People come to the website to view property photos. They were literally smaller than a 256x256 map tile. The size of them should have been increased long ago.

vanderZwan 5y ago
Well, I'm glad that worked out in your case, but that's just getting lucky, no? Sounds like in most other scenarios it would backfire.
- dylan604 5y ago
  That's not the take I got from reading the write up. There was logic in the decision to increase the size that was not difficult to understand. I'm not familiar with what Lighthouse is/does/why/etc, but they described the situation enough for this to make total sense on why it worked. Maybe they are lucky to have people working for them that can work the problem?? That's me stretching to accept lucky in the least as the reasoning.
  - mediumdeviation 5y ago
    It only worked because Google Maps loads images in 250px tiles. If Google Maps decided to update tile size tomorrow, the fix regresses. Arguably the entire map as a whole is one content, and that should be measured as the LCP, but more importantly it shows the measure is just very brittle and easily gamed, as other comments in this thread already attest to.
    - dylan604 5y ago
      Does setting the size inline with width="250" in the embed affect the LCP even if G decides to default to 512px? I'm actually surprised there's not a ?width=250 value in the G embed url
  - Aeolun 5y ago
    > There was logic in the decision to increase the size that was not difficult to understand. I agree, but the logic doesn’t make sense if your goal is ‘optimizing the user experience’ as opposed to ‘optimizing our lighthouse score’.
  - vanderZwan 5y ago
    Apologies, I wasn't specific enough in expressing what I meant. You understood it to mean "they didn't know what they were doing and got lucky", which is a valid interpretation but not what I was going for. They definitely do know what they're doing. What I meant was more along the lines of "the metric to optimize for was no longer strongly correlated to the original goal of improving the user experience. They were lucky that in this scenario optimizing for the metric aligned with an improved user experience."
    - dylan604 5y ago
      Now here's me doing some back bending playing devil's advocate. If the image being optimized for UX resulted in the wacky scoring situation that lowered their ranking in search, would that also not be considered a bad UX if it's harder for users to find you? The team noticed the lower rankings, then investigated. They then realized why the ranking was lower due to the bad LCP. The investigation found that their image optimization was causing the bad LCP, so a quick change on that fixed the bad LCP. So it's not like they were chasing a metric for metric's sake. They did the correct optimization first. So again, I think calling it lucky is a bit harsh in this case. However, I totaly get where you are coming from and know that a lot of places will luck there way into things without ever knowing the why.

rikroots 5y ago

I was surprised that when I tested my canvas library website using the new Lighthouse metrics it scored 92 on performance (and 99 on accessibility, which is the metric I most care about). The website[1] includes a massive animated banner in a <canvas> element which requires loading two large (yet responsive) images alongside processing etc to get the animation playing. The LCP measure was 0.7s Checking web.dev's LCP page[2] explains my luck - Lighthouse does not (currently) consider <canvas> elements, or their asset requirements, when calculating the LCP score. They do include: "<img> elements; <image> elements inside an <svg> element; <video> elements (the poster image is used); an element with a background image loaded via the url() function (as opposed to a CSS gradient); and block-level elements containing text nodes or other inline-level text elements children." I am certainly not suggesting that everyone starts loading their image-heavy banner content into <canvas> elements to help improve their LCP scores as I fully expect Google to address such attempts to game the system at some point in the (near) future. [1] - https://scrawl-v8.rikweb.org.uk/ https://scrawl-v8.rikweb.org.uk/ [2] - https://web.dev/lcp/ https://web.dev/lcp/

jchw 5y ago

Well, they went with a simpler approach earlier, with AMP. AMP couldn’t make any literal guarantees about UX or performance, but in comparison to the mobile experiences reading articles before it, it did seem to often be an improvement. And of course, people hated it, for numerous reasons. You could just make pages faster and less annoying by using plain HTML, obviously. The problem isn’t that anyone knew this, it’s that most of the big publications DGAF, and they were the entire point of AMP as far as I can see. There are many reasons to dislike the way AMP was executed, even if they did eventually rectify many of the serious issues... but the AMP cache concept in combination with the soured reputation probably spelled the end of it no matter what would happen next. AMP for E-mail doesn’t really have any of those problems, so I suspect AMP will probably live on in e-mail. But Google does definitely want to push sites towards better performance and users towards better performing sites; they simply have a mutual interest in this, I don’t think most people would dispute that. Even just in general, but especially when new Android phones are not packing the same punch as new iPhones are... so that scroll jank is going to hit a lot harder. Lighthouse is generally good. It actually discourages a lot of things that people already thought were bad ideas, even ideas from Google. It tends to score plain HTML pages very well, and if it does catch a snag on something you can usually at least work around it. It provides decent diagnostics for how to optimize a web page. It does have some issues, and that will contort people into doing otherwise silly things. But if the website in question chooses a metric in something like Lighthouse in a way that is detrimental to their UX, chances are they just never gave a shit about UX to begin with. While I don’t want to come off as a shill, and I also don’t want to come off as accepting Lighthouse just because it’s “less bad” than AMP, I just think that the “abuse” here is less substantiated. Although there are edge cases where Lighthouse can be silly, it does generally force you to make leaner pages. I think it makes a decent measure of performance. As long as a few points on Lighthouse is not make-or-break for SEO scores, and I doubt it’s that sensitive given the variability per-run, I think this is actually a good thing. Search engine metrics are always imperfect. Anyone familiar with Wikia/Fandom can understand how its subpar content can overshadow much better community-driven websites due to its SEO position, and that is even in spite of what an awful, slow, ad-ridden pile of shit it is. I’d be delighted to see it Page 2’d over Lighthouse scores. But that’s just my feelings on the matter. I’m sure there’s more to it than that.

kmeisthax 5y ago

In this particular case the developer had wondered if chasing Lighthouse was a good idea, but realized that they had already made a similar change on another site for different reasons. Better Lighthouse scores were just a bonus.

linkjuice4all 5y ago

It's been annoying seeing these metrics creep into marketing reports and seeing devs wasting hours "fixing" the issues. Page speed is definitely an issue but I rarely see these stories accompanied by actual user speed data from the site. Google Analytics captures this stuff so you can actually compare pages per session, time on page, scroll distance, etc to paint, load, and render times to "quantify" what users are actually seeing when they use your product.