3 ms·

When I got started with the web 15 years ago, it was advised everywhere not to rely on user agent strings and rely on feature detection instead, and that using

by jraph 4y ago

When I got started with the web 15 years ago, it was advised everywhere not to rely on user agent strings and rely on feature detection instead, and that using the user agent string should be a last resort solution. Today, we are still seeing issues "solved" by switching one's user agents. And here we are reading Akamai whining about user agents getting unreliable. And we are talking unreliable at the minor version and specific platform version level. It's not like we weren't warned ahead of time. I'm sure problems will be sorted by proper http headers, data in handshakes or other things. And they should. Nobody should have to read user agent strings to optimize things, because things should also be optimized for a new, unknown user agent that would support these optimizations.

krono 4y ago
The level of detail that will be available to servers will not be reduced at all, but rather repackaged and split up into separate headers that the server can individually request. The information contained in these headers will likely be more accurate because it's claimed to be safer this way. Whereas today your browser sends the messy but relatively detailed user agent string automatically with each request, after this change it will still send the messy user agent string with each request but with a tiny bit less detail. Google's writers are pretty good at polishing turds, got to give them that!
masklinn 4y ago
> and that using the user agent string should be a last resort solution. In fairness, “last resort solution” means sometimes it is your only solution, when a specific browser fucks up on specific content and you need to work around that specifically.
- jraph 4y ago
  Sure, that belongs to the very few valid use cases. I got an iPad 2 from a relative, I do detect its user agent on my private Invidious instance to send it transpiled/polyfilled JS instead of the original one. Of course it would not be the correct solution if Apple did not forbid other browsers on its hardware, the correct solution would then be to install a recent Firefox version on it. It would also allow a shitload of other stuff to work, like subtitles on fullscreen videos and autoplay on the next video, playback of videos protected by HTTP basic auth, as well as Let's Encrypt SSL certificates. The device's browser should send a "X-I-m-dumb-and-my-manufacturer-likes-to-piss-everybody-off: true" HTTP header to avoid relying on its user-agent though.
  - jeroenhd 4y ago
    FWIW you can get Let's Encrypt working on outdated Apple hardware by manually loading the CA certificate.
    - jraph 4y ago
      I could import the root certificate but instructions for activating it didn't work, they seem to apply on more recent versions of iOS.
      - jeroenhd 4y ago
        I've done it on iOS 9.3.5 and it worked in Safari at the very least. You need to import it first and then activate it from the settings afterwards, I believe. iOS changed the exact procedure a few times so you may need to Google around for the exact steps you need to follow.
bluesmoon 4y ago
I'm not sure how you came to the conclusion that "Akamai is whining" about this. It's an informational blog post about what's happening and what's changing. User Agent strings aren't used for feature detection, they're used for classification. As a developer, when you're trying to fix a bug reported by a customer, it helps to know exactly which browser right down to the patch version that bug shows up in so that you can try and reproduce the bug in the same environment.
- cxr 4y ago
  Odd response—especially the (perversely ironic!) dig in your first sentence. The blog post states: > At Akamai, we use the User-Agent header at the edge and as part of many Akamai products for business logic The post then goes on to describe several things that are expected to break (or would be breaking—if Akamai weren't taking steps on their end) since they rely on the value of client's User-Agent header, and it affects how they respond. It's definitely not just for being used at Akamai to help reproduce bugs in the same environments...
- stickfigure 4y ago
  Then you can ask the customer. You have a relationship with them. Akamai uses user-agent strings in its Bot Manager. They see what specific version of what browser you're running, then check certain characteristics of the request (eg header order) against a database. That isn't going to work anymore. And good riddens. It makes the internet brittle and isn't especially hard to work around anyway.
- jraph 4y ago
  > I'm not sure how you came to the conclusion that "Akamai is whining" about this I largely overstated that. Of course. However, my feeling throughout the entire post is that it was like they were announcing bad news about which they were not too overly about this because they based their optimization strategy on this. I was like "the world told you so".
  - nicj 4y ago
    As the author of the post... :) We don't see this as bad news, quite the opposite in fact. Feature detection (if you're running JavaScript), Client Hints (if you're running code at the edge), etc are all a lot better to use for logic decisions. We've been making changes to rely less on the UA header directly for our Akamai products. Our goal of the post was more around educating our customers, who may not be aware of these changes, and these changes can affect their custom logic, if they depend on the UA.
Beltalowda 4y ago
This is yet another example of good advice that over time gets oversimplified to an 'always' rule, and just becomes silly. "Don't rely on User-Agent" was in response to things like: if (isIE()) useIEThing() elseif (isNetscape()) useNetscapeThing() else alert('unsupported') And that is usually a bad thing to do, since you can always almost replace that with a simple "if ('foo' in window) useFoo()" test or the like. But there are also things that can't be done like this. What if I want to serve the best possible image or video format for a platform? The Accept-Content header isn't really enough for this, aside from that it doesn't really advertise all supported formats on most platforms it also doesn't tell you things like "Firefox 86 enabled AVIF decoding support, and Firefox 100 enabled hardware decoding, but only on Windows". So if someone is using Firefox 101 on Windows: let's serve them an AVIF video, it will work great for them. If they're using Firefox 101 on macOS: maybe use another format because AVIF will eat all their CPU. There's lots of little cases like this where you can't really rely on feature detection. There's a reason User-Agent got replaced by another system which gives the same information: that's because they're useful (IMHO Client Hints are worse by the way and not an improvement at all).
- jraph 4y ago
  For this example I see two possible answers: - Accept-Content has a q-factor weight that can be attached to the types. Browsers should use this correctly to hint at the server their preferred format(s). It should be considered a bug if not. What if I actually tweaked and recompiled my Firefox with a better decoder? Or if the browsers uses a framework like gstreamer / ffmpeg and I installed the right packages for this decoder? You can't know from the UA. Accept-Content cannot possibly list all the supported codecs when the browser accepts a large number of them, but it should at least list the most widespread ones with correct weights. But this leads to my second answer (especially as a very reliable Accept-Content is bad for privacy): - Actually, just use srcset and provide an entry for each format you support and let the browser pick the right one. No need for Accept-Content and CDN magic. It should be the browser's responsibility to know what's supported, what's not, what's best. If not, again, it should be considered a browser bug, the web developer has done their work at this point. The server can't know the best format, only the browser can. I understand that reality is different, but Akamai and YouTube both have leverage on browser vendors to make them fix their bugs / to build standards for this shit. Smaller developers can report bugs too, and I understand that using the UA string is a "worse is better" solution that works around the issues, but we've had years to fix this. A large entity like Akamai should not have relied on UA without preparing cleaner solutions built with the browser vendors, and should not have been caught by surprise by such a change. Something is wrong in this story. Note that I didn't outright reject using the UA string for workarounds, but that's still a last resort which is error prone. Building business logic using the UA string is asking for troubles and we've been known this for a long time now. The proof of this is in the very existence of this article from Akamai. They are "screwed" [1] because they can't really do what they are doing the way they are doing it. [1] I'm sure they are smart and will find solutions. I hope they'll find that building standards with browser vendors is a good solution.
fiedzia 4y ago
> When I got started with the web 15 years ago, it was advised everywhere not to rely on user agent strings and rely on feature detection instead, Which is reasonable advice for a code running in a browser, not for a proxy/CDN (and you don't want proxy inserting it's own js).
- jraph 4y ago
  UA detection in the backend has also been frawned upon, it's not limited to code running in the browser. And a proxy/CDN should not be doing something else than proxying requests and serving files. Workarounds are fine, but that's what they are.
  - fiedzia 4y ago
    To get it working client-side you'd need to change the app, while the whole premise here is that you don't have to (and often can't). One point tracking what UA string are here and what they need is better than expecting every app author to handle this properly.
culopatin 4y ago
My biggest issue with being a solo learner is that unless I specifically look for some information, I don’t know it’s there. Who is we for you? Who was warning everyone? Was I supposed to get a memo?

by jraph 4y ago

krono 4y ago

The level of detail that will be available to servers will not be reduced at all, but rather repackaged and split up into separate headers that the server can individually request. The information contained in these headers will likely be more accurate because it's claimed to be safer this way. Whereas today your browser sends the messy but relatively detailed user agent string automatically with each request, after this change it will still send the messy user agent string with each request but with a tiny bit less detail. Google's writers are pretty good at polishing turds, got to give them that!

masklinn 4y ago

> and that using the user agent string should be a last resort solution. In fairness, “last resort solution” means sometimes it is your only solution, when a specific browser fucks up on specific content and you need to work around that specifically.

jraph 4y ago
Sure, that belongs to the very few valid use cases. I got an iPad 2 from a relative, I do detect its user agent on my private Invidious instance to send it transpiled/polyfilled JS instead of the original one. Of course it would not be the correct solution if Apple did not forbid other browsers on its hardware, the correct solution would then be to install a recent Firefox version on it. It would also allow a shitload of other stuff to work, like subtitles on fullscreen videos and autoplay on the next video, playback of videos protected by HTTP basic auth, as well as Let's Encrypt SSL certificates. The device's browser should send a "X-I-m-dumb-and-my-manufacturer-likes-to-piss-everybody-off: true" HTTP header to avoid relying on its user-agent though.
- jeroenhd 4y ago
  FWIW you can get Let's Encrypt working on outdated Apple hardware by manually loading the CA certificate.
  - jraph 4y ago
    I could import the root certificate but instructions for activating it didn't work, they seem to apply on more recent versions of iOS.
    - jeroenhd 4y ago
      I've done it on iOS 9.3.5 and it worked in Safari at the very least. You need to import it first and then activate it from the settings afterwards, I believe. iOS changed the exact procedure a few times so you may need to Google around for the exact steps you need to follow.

bluesmoon 4y ago

I'm not sure how you came to the conclusion that "Akamai is whining" about this. It's an informational blog post about what's happening and what's changing. User Agent strings aren't used for feature detection, they're used for classification. As a developer, when you're trying to fix a bug reported by a customer, it helps to know exactly which browser right down to the patch version that bug shows up in so that you can try and reproduce the bug in the same environment.

cxr 4y ago
Odd response—especially the (perversely ironic!) dig in your first sentence. The blog post states: > At Akamai, we use the User-Agent header at the edge and as part of many Akamai products for business logic The post then goes on to describe several things that are expected to break (or would be breaking—if Akamai weren't taking steps on their end) since they rely on the value of client's User-Agent header, and it affects how they respond. It's definitely not just for being used at Akamai to help reproduce bugs in the same environments...
stickfigure 4y ago
Then you can ask the customer. You have a relationship with them. Akamai uses user-agent strings in its Bot Manager. They see what specific version of what browser you're running, then check certain characteristics of the request (eg header order) against a database. That isn't going to work anymore. And good riddens. It makes the internet brittle and isn't especially hard to work around anyway.
jraph 4y ago
> I'm not sure how you came to the conclusion that "Akamai is whining" about this I largely overstated that. Of course. However, my feeling throughout the entire post is that it was like they were announcing bad news about which they were not too overly about this because they based their optimization strategy on this. I was like "the world told you so".
- nicj 4y ago
  As the author of the post... :) We don't see this as bad news, quite the opposite in fact. Feature detection (if you're running JavaScript), Client Hints (if you're running code at the edge), etc are all a lot better to use for logic decisions. We've been making changes to rely less on the UA header directly for our Akamai products. Our goal of the post was more around educating our customers, who may not be aware of these changes, and these changes can affect their custom logic, if they depend on the UA.

Beltalowda 4y ago

This is yet another example of good advice that over time gets oversimplified to an 'always' rule, and just becomes silly. "Don't rely on User-Agent" was in response to things like: if (isIE()) useIEThing() elseif (isNetscape()) useNetscapeThing() else alert('unsupported') And that is usually a bad thing to do, since you can always almost replace that with a simple "if ('foo' in window) useFoo()" test or the like. But there are also things that can't be done like this. What if I want to serve the best possible image or video format for a platform? The Accept-Content header isn't really enough for this, aside from that it doesn't really advertise all supported formats on most platforms it also doesn't tell you things like "Firefox 86 enabled AVIF decoding support, and Firefox 100 enabled hardware decoding, but only on Windows". So if someone is using Firefox 101 on Windows: let's serve them an AVIF video, it will work great for them. If they're using Firefox 101 on macOS: maybe use another format because AVIF will eat all their CPU. There's lots of little cases like this where you can't really rely on feature detection. There's a reason User-Agent got replaced by another system which gives the same information: that's because they're useful (IMHO Client Hints are worse by the way and not an improvement at all).

jraph 4y ago
For this example I see two possible answers: - Accept-Content has a q-factor weight that can be attached to the types. Browsers should use this correctly to hint at the server their preferred format(s). It should be considered a bug if not. What if I actually tweaked and recompiled my Firefox with a better decoder? Or if the browsers uses a framework like gstreamer / ffmpeg and I installed the right packages for this decoder? You can't know from the UA. Accept-Content cannot possibly list all the supported codecs when the browser accepts a large number of them, but it should at least list the most widespread ones with correct weights. But this leads to my second answer (especially as a very reliable Accept-Content is bad for privacy): - Actually, just use srcset and provide an entry for each format you support and let the browser pick the right one. No need for Accept-Content and CDN magic. It should be the browser's responsibility to know what's supported, what's not, what's best. If not, again, it should be considered a browser bug, the web developer has done their work at this point. The server can't know the best format, only the browser can. I understand that reality is different, but Akamai and YouTube both have leverage on browser vendors to make them fix their bugs / to build standards for this shit. Smaller developers can report bugs too, and I understand that using the UA string is a "worse is better" solution that works around the issues, but we've had years to fix this. A large entity like Akamai should not have relied on UA without preparing cleaner solutions built with the browser vendors, and should not have been caught by surprise by such a change. Something is wrong in this story. Note that I didn't outright reject using the UA string for workarounds, but that's still a last resort which is error prone. Building business logic using the UA string is asking for troubles and we've been known this for a long time now. The proof of this is in the very existence of this article from Akamai. They are "screwed" [1] because they can't really do what they are doing the way they are doing it. [1] I'm sure they are smart and will find solutions. I hope they'll find that building standards with browser vendors is a good solution.

fiedzia 4y ago

> When I got started with the web 15 years ago, it was advised everywhere not to rely on user agent strings and rely on feature detection instead, Which is reasonable advice for a code running in a browser, not for a proxy/CDN (and you don't want proxy inserting it's own js).

jraph 4y ago
UA detection in the backend has also been frawned upon, it's not limited to code running in the browser. And a proxy/CDN should not be doing something else than proxying requests and serving files. Workarounds are fine, but that's what they are.
- fiedzia 4y ago
  To get it working client-side you'd need to change the app, while the whole premise here is that you don't have to (and often can't). One point tracking what UA string are here and what they need is better than expecting every app author to handle this properly.

culopatin 4y ago

My biggest issue with being a solo learner is that unless I specifically look for some information, I don’t know it’s there. Who is we for you? Who was warning everyone? Was I supposed to get a memo?