6 ms·

Replacing the scroll tracker on GOV.UK

drawfloat 5y ago
I find it hard to believe the devs behind Gov.uk were genuinely considering hard coding all URLs and headings in to FE JS. Surely their mind went to the “window.location” + throttle scroll function solution/“better but not suitable Intersection Observer” options first? Glad they settled on those either way, because adding 200kb of uncompressed URLs and heading text for tracking is madness.
mxz3000 5y ago
I'm consistently really impressed with the UK gov's websites. They're fast, usable and mostly quite intuitive, something that is helped by the fact that they all share a common theme. They support pretty much everything you'd want to do from an admin perspective. This contrasts greatly with US and French government websites I've had the displeasure of using in the past.
- zurgax 5y ago
  Depends which bit you look at in my experience, some of the home office stuff (especially around visas and immigration) still manages to be obscure and is sometimes defective. Perhaps a sufficiently misanthropic department can always get its institutional culture to bleed through the standardised stylesheets.
  - OJFord 5y ago
    I don't know about your specific example, but there's some stuff GDS hasn't touched (yet?) and it shows. The 'modern' stuff that has that consistent design across departments, awesome accessibility etc. is all great IME. Devil's advocate though: I know someone who works in tax (for a private firm of accountants) and hates it. Too dumbed down, and the bar at which they need to reach out to HMRC for guidance or some other resource is just too low. I find it often ends up linking through to actual legislature, so perhaps it's just something in the middle that's missing, the 'technical reference' level.
    - mxz3000 5y ago
      Yeah some stuff hasn't been migrated yet but it's pretty rare in my limited experience. Regarding your devil's advocate example, it's obvious a power user (i.e. tax pro) will have different needs than a layman like me, and it's clear that the sites are built for the latter, which makes way more sense as that's like 99.999% of the users anyways. The power users are probably so rare that just forcing them to phone in makes sense.
    - GrantZvolsky 5y ago
      GDS[1] and HMRC[2] don't have much in common besides the fact that HMRC uses GDS's design system. [1]: https://github.com/alphagov https://github.com/alphagov [2]: https://github.com/hmrc https://github.com/hmrc
      - OJFord 5y ago
        I don't work (and never have) in the public sector, but my understanding was that that's essentially true of any department; that GDS operated like a sort of slightly dictatorial contractor that also provides some common infrastructure and design system etc.? Happy to be corrected, for the purpose of this thread though it was the design system I'm talking about anyway really though.
        GrantZvolsky 5y ago
        It seems that technology-wise, there is more proximity between GDS and the other departments, which is what I was alluding to. This is based on my observation of the various github repositories, with no insight into how the work is organized.
- tails4e 5y ago
  It's nice to see pride in a civil service tech job. For the most part it's absolutely crazy how poor tech is handled in most governments, and nothing is done about it. Source: my sister waited 4 weeks for her civil service work laptop to be fixed. They have contract repair companies, but couldn't be bothered organising it. She ended up driving to the contractor herself after 4 weeks and waited for thr repair. Shocking.
- samwillis 5y ago
  While I agree, venture outside central government to local authority and you still get a shit show of bad websites and services. The other day I paid out yearly fee for our garden waste bin collection. The page takes you to an completely un-styled credit card form with no branding and just the price, no details, then the “confirmation” screen is just white page with a single line “payment received”, nothing else not even a redirect. Oh and no confirmation email. Our local planning site had a bug for six months where when you typed in the search box, focus was lost if you typed to slowly due to the results appearing automatically and steeling focus…
  - diordiderot 5y ago
    It really confuses me. All of "Her Majesty's government's" designs and software are exceptionally well made and freely available. Why can't counties, boroughs, and councils just copy it and add a dash of styling
    - andygh 5y ago
      Having worked in building local government services there's loads of factors of why this hasn't worked (despite GDS trying desperately to make it so). Most of it comes down to money - the vast majority of residents that use Council services are older and/or less likely to have access to a computer or smartphone and/or more likely to need support in accessing these services from someone else. We're pretty good at designing services for these people for the web but frankly, they aren't using them. Some of it is because local government pays poverty wages. Much of the staff I worked with in our IT dept (in a pretty well complimented software development team, it must be said) had been working there 20 years or they are trainees on apprenticeships or fixed-term 1 year contracts. Because of that, there's huge difficulties in adopting agile as a working model, changing technology stacks to ones that the GDS and GOV.UK use is incredibly difficult and recruiting staff that can lead the way on this is an impossibility. Instead, we're relying on outsourcing this to companies like Capita who make some of the worst designed government services I've ever seen. Finally, Councillor's (our budgetholders at the end of the day) just don't see the material benefit of good digital services for their residents. Faced with a year-on-year cut to the grants they get from central government and the poltiics of raising Council Tax, they see IT staff more as a cost centre than as an area for capital development.
- silasdavis 5y ago
  Interacting with gob.es has given my a vast amount of respect for gov.uk.
- userbinator 5y ago
  They also work well in very obscure browsers: https://news.ycombinator.com/item?id=25915313 https://news.ycombinator.com/item?id=25915313
- t3rabytes 5y ago
  usa.gov has gotten quite good thanks to work that USDS has done to standardize design language across the government (and help support agencies that want to adopt it), but getting other agencies to use it has been a slowwwwwwwwwwww process.
hdjjhhvvhga 5y ago
> That would mean adding almost 2,260 URLs to the JavaScript. This might sound reasonable This is one of the many things in today's web I have a problem with.
- dbbk 5y ago
  It actually doesn't sound reasonable, I have no idea what they're talking about.
- hdjjhhvvhga 5y ago
  To all downvoters: please consider that 2260 URLs with 100 characters each make 226 KB. That's 1/4th of a megabyte... On a news site! And for what?! As if infinite scroll wasn't bad enough, you now have the solution you wouldn't have in the first place if you hadn't decided that it's an excellent idea to play with the user's brain by displaying new articles masquerading as the continuation of the previous one. Yes, I know, the ship has sailed, but the whole issue still bothers me.
  - zimpenfish 5y ago
    I've scraped the country pages, and then the internal links, removed all the URLs they won't be scroll-tracking (like '/email-signup'), and turned them into a JS file for a total size of 153742 bytes uncompressed. If you take advantage of the fact that all those country pages share a prefix, you can get the array down to 84423[1] (whilst still knowing the prefix.) Convert it to plain text (because you don't always need it immediatelyas an array) and it comes down to 81399. Then use `lz-string` to compress it and you end up with 26656 bytes. Also since there's only 226 unique prefixes for the links, you could use another level of prefix removal and get it down even further - the JS array is now 38659 bytes, which `lz-string` gets down to 9372 bytes. I'm pretty sure someone smarter than me can get it down even further. [edit: discovered some blank URLs that could be stripped]
    - zimpenfish 5y ago
      For comparison, the crown SVG included on every single page is 2043 bytes.
u2077 5y ago
> but we couldn’t use them because we need to support old browsers like Internet Explorer. Isn’t IE dead?
- spiffytech 5y ago
  GOV.UK likes to make sure that people at the margins can still access their material. See this post from last year about someone using a PlayStation Portable to access their housing benefits: https://news.ycombinator.com/item?id=25915313 https://news.ycombinator.com/item?id=25915313
- chippiewill 5y ago
  IE11 is still supported until June of this year in various Windows SKUs. Longer for certain enterprise customers with extended support. Government websites tend to have very broad support requirements.
  - PoignardAzur 5y ago
    Yeah, but this is just tracking what parts of pages are read. Given how small a share of their traffic IE must be, they could probably skip scroll-tracking on IE.
  - userbinator 5y ago
    Even after official support ends, it will continue to be in use for a long time.
- adamckay 5y ago
  The GOV.UK website will no doubt continue to support IE for a good while yet based on this information [1] from one of the developers of the Government Digital Service, as it's required to be as accessible as possible. [1] - https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiveness-of-simple-html/ https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiven...

I find it hard to believe the devs behind Gov.uk were genuinely considering hard coding all URLs and headings in to FE JS. Surely their mind went to the “window.location” + throttle scroll function solution/“better but not suitable Intersection Observer” options first? Glad they settled on those either way, because adding 200kb of uncompressed URLs and heading text for tracking is madness.

mxz3000 5y ago

I'm consistently really impressed with the UK gov's websites. They're fast, usable and mostly quite intuitive, something that is helped by the fact that they all share a common theme. They support pretty much everything you'd want to do from an admin perspective. This contrasts greatly with US and French government websites I've had the displeasure of using in the past.

zurgax 5y ago
Depends which bit you look at in my experience, some of the home office stuff (especially around visas and immigration) still manages to be obscure and is sometimes defective. Perhaps a sufficiently misanthropic department can always get its institutional culture to bleed through the standardised stylesheets.
- OJFord 5y ago
  I don't know about your specific example, but there's some stuff GDS hasn't touched (yet?) and it shows. The 'modern' stuff that has that consistent design across departments, awesome accessibility etc. is all great IME. Devil's advocate though: I know someone who works in tax (for a private firm of accountants) and hates it. Too dumbed down, and the bar at which they need to reach out to HMRC for guidance or some other resource is just too low. I find it often ends up linking through to actual legislature, so perhaps it's just something in the middle that's missing, the 'technical reference' level.
  - mxz3000 5y ago
    Yeah some stuff hasn't been migrated yet but it's pretty rare in my limited experience. Regarding your devil's advocate example, it's obvious a power user (i.e. tax pro) will have different needs than a layman like me, and it's clear that the sites are built for the latter, which makes way more sense as that's like 99.999% of the users anyways. The power users are probably so rare that just forcing them to phone in makes sense.
  - GrantZvolsky 5y ago
    GDS[1] and HMRC[2] don't have much in common besides the fact that HMRC uses GDS's design system. [1]: https://github.com/alphagov https://github.com/alphagov [2]: https://github.com/hmrc https://github.com/hmrc
    - OJFord 5y ago
      I don't work (and never have) in the public sector, but my understanding was that that's essentially true of any department; that GDS operated like a sort of slightly dictatorial contractor that also provides some common infrastructure and design system etc.? Happy to be corrected, for the purpose of this thread though it was the design system I'm talking about anyway really though.
      - GrantZvolsky 5y ago
        It seems that technology-wise, there is more proximity between GDS and the other departments, which is what I was alluding to. This is based on my observation of the various github repositories, with no insight into how the work is organized.
tails4e 5y ago
It's nice to see pride in a civil service tech job. For the most part it's absolutely crazy how poor tech is handled in most governments, and nothing is done about it. Source: my sister waited 4 weeks for her civil service work laptop to be fixed. They have contract repair companies, but couldn't be bothered organising it. She ended up driving to the contractor herself after 4 weeks and waited for thr repair. Shocking.
samwillis 5y ago
While I agree, venture outside central government to local authority and you still get a shit show of bad websites and services. The other day I paid out yearly fee for our garden waste bin collection. The page takes you to an completely un-styled credit card form with no branding and just the price, no details, then the “confirmation” screen is just white page with a single line “payment received”, nothing else not even a redirect. Oh and no confirmation email. Our local planning site had a bug for six months where when you typed in the search box, focus was lost if you typed to slowly due to the results appearing automatically and steeling focus…
- diordiderot 5y ago
  It really confuses me. All of "Her Majesty's government's" designs and software are exceptionally well made and freely available. Why can't counties, boroughs, and councils just copy it and add a dash of styling
  - andygh 5y ago
    Having worked in building local government services there's loads of factors of why this hasn't worked (despite GDS trying desperately to make it so). Most of it comes down to money - the vast majority of residents that use Council services are older and/or less likely to have access to a computer or smartphone and/or more likely to need support in accessing these services from someone else. We're pretty good at designing services for these people for the web but frankly, they aren't using them. Some of it is because local government pays poverty wages. Much of the staff I worked with in our IT dept (in a pretty well complimented software development team, it must be said) had been working there 20 years or they are trainees on apprenticeships or fixed-term 1 year contracts. Because of that, there's huge difficulties in adopting agile as a working model, changing technology stacks to ones that the GDS and GOV.UK use is incredibly difficult and recruiting staff that can lead the way on this is an impossibility. Instead, we're relying on outsourcing this to companies like Capita who make some of the worst designed government services I've ever seen. Finally, Councillor's (our budgetholders at the end of the day) just don't see the material benefit of good digital services for their residents. Faced with a year-on-year cut to the grants they get from central government and the poltiics of raising Council Tax, they see IT staff more as a cost centre than as an area for capital development.
silasdavis 5y ago
Interacting with gob.es has given my a vast amount of respect for gov.uk.
userbinator 5y ago
They also work well in very obscure browsers: https://news.ycombinator.com/item?id=25915313 https://news.ycombinator.com/item?id=25915313
t3rabytes 5y ago
usa.gov has gotten quite good thanks to work that USDS has done to standardize design language across the government (and help support agencies that want to adopt it), but getting other agencies to use it has been a slowwwwwwwwwwww process.

hdjjhhvvhga 5y ago

> That would mean adding almost 2,260 URLs to the JavaScript. This might sound reasonable This is one of the many things in today's web I have a problem with.

dbbk 5y ago
It actually doesn't sound reasonable, I have no idea what they're talking about.
hdjjhhvvhga 5y ago
To all downvoters: please consider that 2260 URLs with 100 characters each make 226 KB. That's 1/4th of a megabyte... On a news site! And for what?! As if infinite scroll wasn't bad enough, you now have the solution you wouldn't have in the first place if you hadn't decided that it's an excellent idea to play with the user's brain by displaying new articles masquerading as the continuation of the previous one. Yes, I know, the ship has sailed, but the whole issue still bothers me.
- zimpenfish 5y ago
  I've scraped the country pages, and then the internal links, removed all the URLs they won't be scroll-tracking (like '/email-signup'), and turned them into a JS file for a total size of 153742 bytes uncompressed. If you take advantage of the fact that all those country pages share a prefix, you can get the array down to 84423[1] (whilst still knowing the prefix.) Convert it to plain text (because you don't always need it immediatelyas an array) and it comes down to 81399. Then use `lz-string` to compress it and you end up with 26656 bytes. Also since there's only 226 unique prefixes for the links, you could use another level of prefix removal and get it down even further - the JS array is now 38659 bytes, which `lz-string` gets down to 9372 bytes. I'm pretty sure someone smarter than me can get it down even further. [edit: discovered some blank URLs that could be stripped]
  - zimpenfish 5y ago
    For comparison, the crown SVG included on every single page is 2043 bytes.

u2077 5y ago

> but we couldn’t use them because we need to support old browsers like Internet Explorer. Isn’t IE dead?

spiffytech 5y ago
GOV.UK likes to make sure that people at the margins can still access their material. See this post from last year about someone using a PlayStation Portable to access their housing benefits: https://news.ycombinator.com/item?id=25915313 https://news.ycombinator.com/item?id=25915313
chippiewill 5y ago
IE11 is still supported until June of this year in various Windows SKUs. Longer for certain enterprise customers with extended support. Government websites tend to have very broad support requirements.
- PoignardAzur 5y ago
  Yeah, but this is just tracking what parts of pages are read. Given how small a share of their traffic IE must be, they could probably skip scroll-tracking on IE.
- userbinator 5y ago
  Even after official support ends, it will continue to be in use for a long time.
adamckay 5y ago
The GOV.UK website will no doubt continue to support IE for a good while yet based on this information [1] from one of the developers of the Government Digital Service, as it's required to be as accessible as possible. [1] - https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiveness-of-simple-html/ https://shkspr.mobi/blog/2021/01/the-unreasonable-effectiven...