22 ms·

YouTubePluginReplacement.cpp: YouTube-specific code in WebKit

245 points by goranmoomin 4y ago

eimrine 4y ago
Could you tell those who are not familiar with WebKit what this code actually does?
- hadrien01 4y ago
  When the first iPhone was released, YouTube used Flash instead of HTML5. Safari had special code to play the video with the native player instead of displaying a blank square. I suppose (from the flash-specific detection in the supportsMIMEType method) that this code was in charge of that logic.
  - JonathonW 4y ago
    Also the youTubeFlashPluginReplacementEnabled settings property that it looks for to check if it’s enabled seems like a strong hint.
  - londons_explore 4y ago
    Note that while the code was written in 2014 (well after the iphone was released), someone tried to remove the code this year[1], and gave up because some sites now rely on this hack. [1]: https://github.com/WebKit/WebKit/commit/049188a35af6c67fa30c0bbec313afc5c3bda8e6 https://github.com/WebKit/WebKit/commit/049188a35af6c67fa30c...
    - oefrha 4y ago
      Of course, all observable behaviors of your system will be depended on by somebody.
      - Anunayj 4y ago
        Relevant xkcd: https://xkcd.com/1172/ https://xkcd.com/1172/
    - bdash 4y ago
      The code was added to the public WebKit source tree, but it wasn't written at the time. It was upstreamed from the internal fork of WebKit that Apple used for iOS.
      - saagarjha 4y ago
        Do you know how long this fork existed?
        bdash 4y ago
        Sorry, I don't recall.
    - thrdbndndn 4y ago
      It didn't specify which websites, now I'm curious.
DaleCurtis 4y ago
Chrome has something similar here: https://source.chromium.org/chromium/chromium/src/+/main:chrome/renderer/media/flash_embed_rewrite.cc;l=10;drc=82b01954552a812c61cf7bea4709e1ab9ca82e58 https://source.chromium.org/chromium/chromium/src/+/main:chr... We accept contributions from other sites (e.g., Vimeo, Dailymotion, etc) who wanted to replace their long tail of in the wild embeds prior to Flash removal.
- simonsarris 4y ago
  Thanks for your work, Dale. IMO this was good practice to keep the web running smoothly at the time. It is reminiscent of Windows 95's specific code to see if SimCity is running, and allocate memory differently if it was. > Jon Ross, who wrote the original version of SimCity for Windows 3.x, told me that he accidentally left a bug in SimCity where he read memory that he had just freed. Yep. It worked fine on Windows 3.x, because the memory never went anywhere. Here’s the amazing part: On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the memory allocator in a special mode that doesn’t free memory right away. That’s the kind of obsession with backward compatibility that made people willing to upgrade to Windows 95. from: https://www.joelonsoftware.com/2000/05/24/strategy-letter-ii-chicken-and-egg-problems/ https://www.joelonsoftware.com/2000/05/24/strategy-letter-ii...
  - tantalor 4y ago
    Yeah why fix bugs when you can just enshrine the bug as a feature. Backwards compatibility like this is a mistake.
    - bigyikes 4y ago
      What you call a “mistake” is actually just having empathy for the user. User experience is king. It’s not tech debt, it’s a feature.
      - tcmb 4y ago
NegativeLatency 4y ago
The video in one of the test cases is Steve Jobs commencement address https://www.youtube.com/v/UF8uR6Z6KLc https://www.youtube.com/v/UF8uR6Z6KLc
- tcmb 4y ago
  Probably the Apple-internal version of Rickrolling.
chrismorgan 4y ago
I believe this is about allowing this kind of markup (that exists on old web pages that haven’t been updated for a long time) to still work: <embed src="http://www.youtube.com/v/dQw4w9WgXcQ"></embed> … by roughly translating it into <iframe src="http://www.youtube.com/embed/dQw4w9WgXcQ"></iframe> Probably also the <object> equivalent, maybe only that and not <embed>, I’m not sure. I never had a great deal to do with those two tags. I don’t know why it has to be done this way. I’d have thought that without this extra code it’d roughly fall back to being an iframe on the original URL, and then YouTube could just check if it’s in a frame and behave as an embed instead of the main UI. But there’s probably a reason why they’ve all done it this kind of way.
- pknerd 4y ago
  Am I the only one who actually visited this Youtube link by copying and pasting in address bar?
  - 10000truths 4y ago
    Anyone who’s been on the Internet long enough will have wisened up to the fabled YouTube link ending in in XcQ.
    - neodypsis 4y ago
      > Anyone who’s been on the Internet long enough will have wisened up to the fabled YouTube link ending in in XcQ "We've known each other for so long"
    - chrismorgan 4y ago
      Funny, it’s the dQw that I recognise.
    - Akronymus 4y ago
      Thats why I use this one: https://www.youtube.com/watch?v=oT3mCybbhf0 https://www.youtube.com/watch?v=oT3mCybbhf0
  - 4y ago
shadowgovt 4y ago
I'm not sure if it's still there, but for a long time there was a chunk of code in the Mozilla code base that hunted for several browser extensions in Windows every time the browser started up and did some patch up work in the file system to make sure that those extensions could be found in the browser because some older installers would dump them in non-standard places. This was to avoid the issue that if the user had already installed the application with the corresponding extension and it was working in Internet Explorer, it wouldn't automatically work in Firefox without this patch up process. And in that case, it wasn't the extension that was broken from the user's point of view... It was Firefox. Some technologies are so fundamental to the web experience that when they break, the user perceives the browser to be broken, not that technology.
- eastbound 4y ago
  I bet the AskBar was always in the perfect standard place for Firefox to find it, despite being run by Oracle (Yes, Oracle installers for Java included the AskBar. A crime in my opinion).
danielovichdk 4y ago
Most beautiful cpp code I have seen in many years
- russellbeattie 4y ago
  I was just about to make this same exact comment! I was half way down the page and thought, "Wait... What language is this again?" I had to scroll back up to confirm.
  - JeffRats 4y ago
    Clean and readable, more like C than meta filled C++ with templates.
    - olliej 4y ago
      Nah, it’s just having professional coding and review standards that are very heavily enforced. This particular code is also aided by being fairly simple in function of course. In my experience C code is generally worse than C++, as a lot of what large C projects do is stuff that C++ does automatically, and with more safety. That said for the most squirrely code in webkit you need to look to JSC where the perf critical and machine abusing code is stuck (though obviously minimized as much as possible). My most terrible contribution there is the parser which uses macros and templates together (parsing JS is extremely perf critical in real world page loading because the parser often has to deal with megs of code, all of which has to be parsed before you can consider executing it and in many places page loading is blocked on said execution)
- jyap 4y ago
  Probably because it’s mostly if statements, neat functions and string translation.
  - ravenstine 4y ago
    It's almost as if the reason C++ can suck is actually because most developers write code that's incomprehensible.
evilpie 4y ago
Firefox will also rewrite YouTube embeds: https://searchfox.org/mozilla-central/rev/c6d17806faef8708b6511e9849132463b7ea6c37/dom/base/nsObjectLoadingContent.cpp#846 https://searchfox.org/mozilla-central/rev/c6d17806faef8708b6.... Firefox also ships a "system" add-on (https://github.com/mozilla-extensions/webcompat-addon https://github.com/mozilla-extensions/webcompat-addon) that has interventions and user-agent overrides for many websites. See for yourself in Firefox on about:compat.
saagarjha 4y ago
Most platform vendors have this in some form or the other. WebKit has far more here, for example: https://github.com/WebKit/WebKit/blob/main/Source/WebCore/page/Quirks.cpp https://github.com/WebKit/WebKit/blob/main/Source/WebCore/pa.... These are cool and all for when you want software to keep working, but if you're the person writing these, the one thing you should absolutely make sure to do is reach out to the authors of the software you're working around to see if they are willing to fix their code. Most of the time they are! But I invariable see code go into the OS/browser/whatever and engineers bemoaning how they have to work super hard on backcompat, and I'm like…"ok, but when did you send us a bug?"
leeoniya 4y ago
hehe, now let's see all the WebKit-specific code in YouTube :D
orblivion 4y ago
Some people here are complaining that this is a hack, technical debt, etc. I thought the point would be the political one, as it were. YouTube.com gets special treatment in a major web browser, whereas Sal's discount video website has to conform to standards like all the other plebs. It's an erosion (however minor, by itself) of the idea of an open standard.
- scarface74 4y ago
  Apple has a list of websites with quirks that don’t support its “generate strong passwords” feature even though there is HTML spec where you can specify the types of passwords you support. https://github.com/apple/password-manager-resources https://github.com/apple/password-manager-resources Many of these sites are quite obscure.
  - dopa42365 4y ago
    https://github.com/apple/password-manager-resources/blob/main/quirks/password-rules.json https://github.com/apple/password-manager-resources/blob/mai... For being "quite obscure", I've at least heard of most of these sites before. Banks with "maxlength: 8", you love to see it.
    - cuteboy19 4y ago
      QQ: why do they embed the rules in a string and not just regular json?
    - scarface74 4y ago
      I just actually looked at the JSON, I just assumed the rule was going to be defined as regex, not a human readable pattern.
    - d12bb 4y ago
      I just love that apple.com itself is included there.
  - yreg 4y ago
cush 4y ago
Odd that a quarter of the code is parsing the qsps
jdmnd 4y ago
This is interesting. It looks like there was even some discussion of making this behaviour part of the standard, since it's implemented by all of the major browsers. https://github.com/whatwg/html/issues/2390 https://github.com/whatwg/html/issues/2390

245 points by goranmoomin 4y ago

eimrine 4y ago

Could you tell those who are not familiar with WebKit what this code actually does?

hadrien01 4y ago
When the first iPhone was released, YouTube used Flash instead of HTML5. Safari had special code to play the video with the native player instead of displaying a blank square. I suppose (from the flash-specific detection in the supportsMIMEType method) that this code was in charge of that logic.
- JonathonW 4y ago
  Also the youTubeFlashPluginReplacementEnabled settings property that it looks for to check if it’s enabled seems like a strong hint.
- londons_explore 4y ago
  Note that while the code was written in 2014 (well after the iphone was released), someone tried to remove the code this year[1], and gave up because some sites now rely on this hack. [1]: https://github.com/WebKit/WebKit/commit/049188a35af6c67fa30c0bbec313afc5c3bda8e6 https://github.com/WebKit/WebKit/commit/049188a35af6c67fa30c...
  - oefrha 4y ago
    Of course, all observable behaviors of your system will be depended on by somebody.
    - Anunayj 4y ago
      Relevant xkcd: https://xkcd.com/1172/ https://xkcd.com/1172/
  - bdash 4y ago
    The code was added to the public WebKit source tree, but it wasn't written at the time. It was upstreamed from the internal fork of WebKit that Apple used for iOS.
    - saagarjha 4y ago
      Do you know how long this fork existed?
      - bdash 4y ago
        Sorry, I don't recall.
  - thrdbndndn 4y ago
    It didn't specify which websites, now I'm curious.

DaleCurtis 4y ago

Chrome has something similar here: https://source.chromium.org/chromium/chromium/src/+/main:chrome/renderer/media/flash_embed_rewrite.cc;l=10;drc=82b01954552a812c61cf7bea4709e1ab9ca82e58 https://source.chromium.org/chromium/chromium/src/+/main:chr... We accept contributions from other sites (e.g., Vimeo, Dailymotion, etc) who wanted to replace their long tail of in the wild embeds prior to Flash removal.

simonsarris 4y ago
Thanks for your work, Dale. IMO this was good practice to keep the web running smoothly at the time. It is reminiscent of Windows 95's specific code to see if SimCity is running, and allocate memory differently if it was. > Jon Ross, who wrote the original version of SimCity for Windows 3.x, told me that he accidentally left a bug in SimCity where he read memory that he had just freed. Yep. It worked fine on Windows 3.x, because the memory never went anywhere. Here’s the amazing part: On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the memory allocator in a special mode that doesn’t free memory right away. That’s the kind of obsession with backward compatibility that made people willing to upgrade to Windows 95. from: https://www.joelonsoftware.com/2000/05/24/strategy-letter-ii-chicken-and-egg-problems/ https://www.joelonsoftware.com/2000/05/24/strategy-letter-ii...
- tantalor 4y ago
  Yeah why fix bugs when you can just enshrine the bug as a feature. Backwards compatibility like this is a mistake.
  - bigyikes 4y ago
    What you call a “mistake” is actually just having empathy for the user. User experience is king. It’s not tech debt, it’s a feature.
    - tcmb 4y ago

NegativeLatency 4y ago

The video in one of the test cases is Steve Jobs commencement address https://www.youtube.com/v/UF8uR6Z6KLc https://www.youtube.com/v/UF8uR6Z6KLc

tcmb 4y ago
Probably the Apple-internal version of Rickrolling.

chrismorgan 4y ago

I believe this is about allowing this kind of markup (that exists on old web pages that haven’t been updated for a long time) to still work: <embed src="http://www.youtube.com/v/dQw4w9WgXcQ"></embed> … by roughly translating it into <iframe src="http://www.youtube.com/embed/dQw4w9WgXcQ"></iframe> Probably also the <object> equivalent, maybe only that and not <embed>, I’m not sure. I never had a great deal to do with those two tags. I don’t know why it has to be done this way. I’d have thought that without this extra code it’d roughly fall back to being an iframe on the original URL, and then YouTube could just check if it’s in a frame and behave as an embed instead of the main UI. But there’s probably a reason why they’ve all done it this kind of way.

pknerd 4y ago
Am I the only one who actually visited this Youtube link by copying and pasting in address bar?
- 10000truths 4y ago
  Anyone who’s been on the Internet long enough will have wisened up to the fabled YouTube link ending in in XcQ.
  - neodypsis 4y ago
    > Anyone who’s been on the Internet long enough will have wisened up to the fabled YouTube link ending in in XcQ "We've known each other for so long"
  - chrismorgan 4y ago
    Funny, it’s the dQw that I recognise.
  - Akronymus 4y ago
    Thats why I use this one: https://www.youtube.com/watch?v=oT3mCybbhf0 https://www.youtube.com/watch?v=oT3mCybbhf0
- 4y ago

shadowgovt 4y ago

I'm not sure if it's still there, but for a long time there was a chunk of code in the Mozilla code base that hunted for several browser extensions in Windows every time the browser started up and did some patch up work in the file system to make sure that those extensions could be found in the browser because some older installers would dump them in non-standard places. This was to avoid the issue that if the user had already installed the application with the corresponding extension and it was working in Internet Explorer, it wouldn't automatically work in Firefox without this patch up process. And in that case, it wasn't the extension that was broken from the user's point of view... It was Firefox. Some technologies are so fundamental to the web experience that when they break, the user perceives the browser to be broken, not that technology.

eastbound 4y ago
I bet the AskBar was always in the perfect standard place for Firefox to find it, despite being run by Oracle (Yes, Oracle installers for Java included the AskBar. A crime in my opinion).

danielovichdk 4y ago

Most beautiful cpp code I have seen in many years

russellbeattie 4y ago
I was just about to make this same exact comment! I was half way down the page and thought, "Wait... What language is this again?" I had to scroll back up to confirm.
- JeffRats 4y ago
  Clean and readable, more like C than meta filled C++ with templates.
  - olliej 4y ago
    Nah, it’s just having professional coding and review standards that are very heavily enforced. This particular code is also aided by being fairly simple in function of course. In my experience C code is generally worse than C++, as a lot of what large C projects do is stuff that C++ does automatically, and with more safety. That said for the most squirrely code in webkit you need to look to JSC where the perf critical and machine abusing code is stuck (though obviously minimized as much as possible). My most terrible contribution there is the parser which uses macros and templates together (parsing JS is extremely perf critical in real world page loading because the parser often has to deal with megs of code, all of which has to be parsed before you can consider executing it and in many places page loading is blocked on said execution)
jyap 4y ago
Probably because it’s mostly if statements, neat functions and string translation.
- ravenstine 4y ago
  It's almost as if the reason C++ can suck is actually because most developers write code that's incomprehensible.

evilpie 4y ago

Firefox will also rewrite YouTube embeds: https://searchfox.org/mozilla-central/rev/c6d17806faef8708b6511e9849132463b7ea6c37/dom/base/nsObjectLoadingContent.cpp#846 https://searchfox.org/mozilla-central/rev/c6d17806faef8708b6.... Firefox also ships a "system" add-on (https://github.com/mozilla-extensions/webcompat-addon https://github.com/mozilla-extensions/webcompat-addon) that has interventions and user-agent overrides for many websites. See for yourself in Firefox on about:compat.

saagarjha 4y ago

Most platform vendors have this in some form or the other. WebKit has far more here, for example: https://github.com/WebKit/WebKit/blob/main/Source/WebCore/page/Quirks.cpp https://github.com/WebKit/WebKit/blob/main/Source/WebCore/pa.... These are cool and all for when you want software to keep working, but if you're the person writing these, the one thing you should absolutely make sure to do is reach out to the authors of the software you're working around to see if they are willing to fix their code. Most of the time they are! But I invariable see code go into the OS/browser/whatever and engineers bemoaning how they have to work super hard on backcompat, and I'm like…"ok, but when did you send us a bug?"

leeoniya 4y ago

hehe, now let's see all the WebKit-specific code in YouTube :D

orblivion 4y ago

Some people here are complaining that this is a hack, technical debt, etc. I thought the point would be the political one, as it were. YouTube.com gets special treatment in a major web browser, whereas Sal's discount video website has to conform to standards like all the other plebs. It's an erosion (however minor, by itself) of the idea of an open standard.

scarface74 4y ago
Apple has a list of websites with quirks that don’t support its “generate strong passwords” feature even though there is HTML spec where you can specify the types of passwords you support. https://github.com/apple/password-manager-resources https://github.com/apple/password-manager-resources Many of these sites are quite obscure.
- dopa42365 4y ago
  https://github.com/apple/password-manager-resources/blob/main/quirks/password-rules.json https://github.com/apple/password-manager-resources/blob/mai... For being "quite obscure", I've at least heard of most of these sites before. Banks with "maxlength: 8", you love to see it.
  - cuteboy19 4y ago
    QQ: why do they embed the rules in a string and not just regular json?
  - scarface74 4y ago
    I just actually looked at the JSON, I just assumed the rule was going to be defined as regex, not a human readable pattern.
  - d12bb 4y ago
    I just love that apple.com itself is included there.
- yreg 4y ago

cush 4y ago

Odd that a quarter of the code is parsing the qsps

jdmnd 4y ago

This is interesting. It looks like there was even some discussion of making this behaviour part of the standard, since it's implemented by all of the major browsers. https://github.com/whatwg/html/issues/2390 https://github.com/whatwg/html/issues/2390