4 ms·

Solving Different Problems

42 points by pmarin 14y ago

DanWaterworth 14y ago
> it has to convert YUV data to the RGB colorspace and combine the image with other Flash elements. The screen has red, green and blue pixels, either way a video is played, the colours have to be converted into RGB.
- ebassi 14y ago
  not every video frame is encoded in RGB. actually, very few transports use that. the in-memory layout of a YUV-encoded frame is different than the one of a RGB one. it needs conversion, which implies an intermediate copy, as well as traversing a buffer that can go up to 1080 rows. most video players (and Flash on Windows and MacOS), these days, use shaders and multi-texturing to do that conversion on the GPU; it removes a copy, because you just push the YUV frame as it is to the GPU - and your CPU can go back to a low C state. I think Flash on Linux does this as well, on nVidia: the blog post linked is from 2010. the way Adobe detect capabilities is, in itself, hilariously bad - instead of checking the GL required GL extensions like any normal people, they do a check on the GL vendor string. the Mesa guys pointed that out, but I doubt they ever received a response.
  - darkestkhan 14y ago
    My system isn't shader capable and still playing HTML5 video tag consumes only ~30% of CPU while playing flash ~90%. There is no excuse for that - I could understand higher CPU usage, but not by factor of 3.
    - novalis 14y ago
      Are you on Linux ? That is the only plat that I see taking higher cpu usage while playing viedo in flash. Both windows and osx now play flash with little cpu usage.
      - darkestkhan 14y ago
        Yes, I'm on Linux - but it still doesn't explain why flash is consuming so much CPU (I'm using VESA drivers for graphics)
  - antihero 14y ago
    Why don't they encode the elements to YUV once, and then flip to YUV (so not needing to convert each time), as opposed to converting the video to RGB?
    - 0x0 14y ago
      I _think_ you lose a _lot_ of color going from RGB->YUV, also you are just replacing one expensive color space conversion with another.
ebassi 14y ago
translation: we don't want to do colorspace conversion on the GPU because it's hard. let's go shopping instead!
- Irfaan 14y ago
  I only skimmed the article, but that strikes me as a "mistranslation". Adobe is emphasizing that they have to do more compositing logic, so they they need an RGB version of the video frame on the CPU side. If they did the colorspace conversation on the GPU, they'd have to pull the converted image back and incur the latency hit. Apparently, they see less latency in doing the conversation on the CPU, and have made the call to trade processor efficiency for latency. To be clear: YUV colorspace conversion on a GPU is really damn simple. I have a ~30 line shader that does it from my video processing codebase. But I can take the hit - I do a substantial amount of image manipulation using shaders on the GPU, and mask the read latency with heavy multi-threading. A Flash application doesn't have this luxury. But this discussion is largely academic - if you're writing a Flash based video player that doesn't need the flexibility of Flash's full compliment of image manipulation and composting functionality, you'd be using their StageVideo API - an API that does do all the video work on the GPU. This API was introduced in Flash 10.2, which came out after this article from 2010.
jrockway 14y ago
the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem Really? The Internet at large is able to unanimously declare things, and one of those things is that it appreciates Flash's solution to the video problem? Tell me more...
- vacri 14y ago
  To be fair, when the article was written in January 2010, the only widespread solution to video in browsers was flash.
  - TazeTSchnitzel 14y ago
    Same with audio, which is why notification sounds on Facebook and such were always done with Flash (not now).
    - lazugod 14y ago
      Facebook has sounds?
      - TazeTSchnitzel 14y ago
        Maybe you're using noscript. If you receive a chat message and you're in another tab, a sound plays.
  - jrockway 14y ago
    Wasn't Netflix Silverlight then?
    - vacri 14y ago
      As an Australian, Netflix was never part of the web that I could see :)
Liongadev 14y ago
While watching streams of Olympia in both HTML5 (on Chrome) and Flash i found HTML5 to take nearly twice as much CPU then Flash. Might be due to the implementation of the sites (german broadcasters ard & zdf) but flash works pretty awesome for me.
- badboy 14y ago
  Besides that it's a shame that the option to just get the stream running in VLC is such a hassle (with VLC I get much lower CPU usage + the ability to freely move and resize the video and place it somewhere on my second screen, even fullscreen)
zacharyvoase 14y ago
Wow. A tip to anyone reading this: if you ever find yourself explaining away poor—nay, pathological—performance with anything other than ‘we made some mistakes in the past and we’re doing what we can to correct them’, please take a good hard look at yourself and consider exiting the software industry. That’s all I have to say. Edit: that wasn’t all I had to say. > What about Mac? I’m not sure but my Mac colleagues have mentioned something about Apple not making their hardware decoding APIs available to applications Well why didn’t you just go ahead and ask them before writing a blog post on company letterhead?! Is the corporate structure here so screwed up that you can’t actually do that, or are you just lazy?
- pi18n 14y ago
  Doubly so if your product crashes even half as often as Flash. That they cannot seem to code for stability doesn't give me any confidence in their ability to code for speed. I heard that they refused to work with Apple to improve its performance on Mac, and that was a core reason why they got excluded from the iOS platform -- searching for this I came up with an explanation directly from Apple: "Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash. We have been working with Adobe to fix these problems, but they have persisted for several years now. We don’t want to reduce the reliability and security of our iPhones, iPods and iPads by adding Flash."
  - 0x0 14y ago
    You must have heard wrong, and it's the other way around. Apple did not open their video acceleration APIs until they released OS X 10.6.3 circla mid-2010. http://xbmc.org/davilla/2010/05/03/osx-gets-h-264-accelleration/ http://xbmc.org/davilla/2010/05/03/osx-gets-h-264-accellerat...
    - pi18n 14y ago
      If that's the major reason preventing them from being performant, then I misunderstood the reason for crashing and the reason for poor performance to be the same. Apple should not be complaining about CPU usage. That doesn't excuse Adobe from making buggy plugins.
illicium 14y ago
> Flash Player solves a different problem: It plays linear media files from start to finish while combining the video with a wide array of graphical and interactive elements (buttons, bitmaps, vector graphics, filters) And therein lies the problem with Flash video: dozens of video player implementations, each frustratingly different and user unfriendly than the next. People are building custom chrome on top of HTML5 video too, unfortunately, but at least it's standardized (and semantic) which allows users to consume web video with any interface they want.
jhack 14y ago
Wonder if Linux video drivers have added the ability to retrieve the decoded video data in the two and a half years since this blog post was made.
nicky0 14y ago
This is a blog post from January 2010. I think a lot of the details are out of date. (Mac hardware decoding support, for example.)
tluyben2 14y ago
I don't know if the explanation is accurate or just their own product defense, however (long) after playing a video (and not something else in flash), I usually go into top and kill the Chrome flash process manually as it's a) taking an insane amount of memory b) taking a lot of CPU for no reason (I'm not watching the video anymore and closed the page). Seems there is a ton of leakage going on after playing a video. So even though the story might to explain the actual playing of the video, it doesn't explain the molestation of my i7/8gb/ssd machine AFTER I closed (the only) Flash window. Not to mention the crashes.
lispm 14y ago
The message that I take is that some companies use Flash as the wrong tool to solve a problem which I'm not interested in. I want to see videos with minimum overhead. I don't want to be bothered with ads, poor performance, poor user interfaces, poor security, ... all which I get through Flash.
- lmm 14y ago
  Given the options available to them (roughly: expensive real video platform, expensive quicktime video platform, cheap content creation platform (in the form of flash) that can do video), those companies (and let's be honest, there's really only one that matters, youtube) made a perfectly reasonable decision. You might moan about flash interfaces, but unless you're willing to pay more (or, equivalently, view more ads) for a real, performant video interface (which was a genuine option for a while - e.g. the now-defunct stage6), you can't blame them for making that choice. (I feel it's a real failure of vision in the open-source community that there wasn't a free theora (say) plugin available for all browsers, and server-side video hosting platform, when youtube was getting started.)
  - lispm 14y ago
    h264 has been a standard for a long time and performs very well on current hardware (many machines have hardware acceleration for it). Google has wasted terawatts of electricity due to flash.
    - lmm 14y ago
      Again, if users cared enough they would vote with their feet. It seems like no-hassle, low-ads video playback is worth more to us than playing it more efficiently.
0x0 14y ago
This is an extremely outdated article which was posted before they added "StageVideo", an API that fixes this problem 100%. http://www.adobe.com/devnet/flashplayer/stagevideo.html http://www.adobe.com/devnet/flashplayer/stagevideo.html
- jessriedel 14y ago
  So is this evidence that the blog author was full of it when he wrote this?
  - 0x0 14y ago
    Not really, StageVideo has a bunch of limitations in the compositing pipeline, but that doesn't really matter if all you want to do is build a 2D movie player. Anyways, I guess Adobe changed their mind later, and decided they wanted to solve both of the "different problems" mentioned in the blog post. (Remember, Flash used to be all about vector animations. Sometimes it'd seem like the ubiquitous flash-as-a-video-player thing happened almost by accident; it turns out the video support (a relatively small feature in the whole flash featureset) was good enough to bring "universal" video to the web.)
rdsubhas 14y ago
tl;dr - we accidentally a whole software video decoder to show some circles and rectangles at the bottom. and so that you can haz internet when playing video.
alvarosm 14y ago
Ridiculous. YUV->RGB conversion + drawing a couple of things is negligibly cheap even if done in CPU. What a pathetic excuse.
- acdha 14y ago
  Not to mention that they were flat out lying when they claimed other players don't do this. Anyone who remembers when Mac devs were hand-tuning AltiVec implementations of YUV-RGB routines knows that Adobe was just choosing not to spend the money.
- greyfade 14y ago
  Yeah, that's pretty dubious. I was under the impression that all video players do that anyway - that video is encoded in the YUV color space and always converted to RGB on playback. To blame that for their poor performance suggests that their YUV-RGB conversion code is among the slowest in the industry.
its_so_on 14y ago
If only there were a way to let a plugin give VLC a URL to open and overlay over a specific part of your screen until you close the tab. Nah, that's crazy talk. Who would want to watch a high-quality hardware-accelerated local video that gets deleted when you close the tab, from the familiar interface of a web browser? Crazy talk, crazy talk.
- inportb 14y ago
  There is. http://www.videolan.org/doc/play-howto/en/ch04.html#id591206 http://www.videolan.org/doc/play-howto/en/ch04.html#id591206 Then again, I don't use it because I couldn't get any seek UI to appear.
brudgers 14y ago
"The Flash Player solves a different problem than your favorite video player." That problem is largely tracking user behavior across the internet.
- ThisIsADogHello 14y ago
  Don't forget that Flash also solves the problem of denying access to your selected media until unskippable ads have played, or overlaying said ads on top of the media you're trying to watch.
lincolnwebs 14y ago
The condescension in that post is galling. "Try to keep up here." "Again, say it with me:" "Please tell me you know people who aren’t as tech savvy as you. If you don’t, get to know some in order to maintain proper perspective." A lesson in how to not communicate a technical idea.
rocky1138 14y ago
This comment on the blog itself perfectly sums up my feeling about the article and its content, so I'll quote it here: "Again, say it with me… I’ve been reading this blog since the beginning. I always thought you were a little condescending, but you seemed knowledgeable enough and I thought you were doing good work.However, you can say this with me: Eat shit."

42 points by pmarin 14y ago

> it has to convert YUV data to the RGB colorspace and combine the image with other Flash elements. The screen has red, green and blue pixels, either way a video is played, the colours have to be converted into RGB.

ebassi 14y ago
not every video frame is encoded in RGB. actually, very few transports use that. the in-memory layout of a YUV-encoded frame is different than the one of a RGB one. it needs conversion, which implies an intermediate copy, as well as traversing a buffer that can go up to 1080 rows. most video players (and Flash on Windows and MacOS), these days, use shaders and multi-texturing to do that conversion on the GPU; it removes a copy, because you just push the YUV frame as it is to the GPU - and your CPU can go back to a low C state. I think Flash on Linux does this as well, on nVidia: the blog post linked is from 2010. the way Adobe detect capabilities is, in itself, hilariously bad - instead of checking the GL required GL extensions like any normal people, they do a check on the GL vendor string. the Mesa guys pointed that out, but I doubt they ever received a response.
- darkestkhan 14y ago
  My system isn't shader capable and still playing HTML5 video tag consumes only ~30% of CPU while playing flash ~90%. There is no excuse for that - I could understand higher CPU usage, but not by factor of 3.
  - novalis 14y ago
    Are you on Linux ? That is the only plat that I see taking higher cpu usage while playing viedo in flash. Both windows and osx now play flash with little cpu usage.
    - darkestkhan 14y ago
      Yes, I'm on Linux - but it still doesn't explain why flash is consuming so much CPU (I'm using VESA drivers for graphics)
- antihero 14y ago
  Why don't they encode the elements to YUV once, and then flip to YUV (so not needing to convert each time), as opposed to converting the video to RGB?
  - 0x0 14y ago
    I _think_ you lose a _lot_ of color going from RGB->YUV, also you are just replacing one expensive color space conversion with another.

ebassi 14y ago

translation: we don't want to do colorspace conversion on the GPU because it's hard. let's go shopping instead!

Irfaan 14y ago
I only skimmed the article, but that strikes me as a "mistranslation". Adobe is emphasizing that they have to do more compositing logic, so they they need an RGB version of the video frame on the CPU side. If they did the colorspace conversation on the GPU, they'd have to pull the converted image back and incur the latency hit. Apparently, they see less latency in doing the conversation on the CPU, and have made the call to trade processor efficiency for latency. To be clear: YUV colorspace conversion on a GPU is really damn simple. I have a ~30 line shader that does it from my video processing codebase. But I can take the hit - I do a substantial amount of image manipulation using shaders on the GPU, and mask the read latency with heavy multi-threading. A Flash application doesn't have this luxury. But this discussion is largely academic - if you're writing a Flash based video player that doesn't need the flexibility of Flash's full compliment of image manipulation and composting functionality, you'd be using their StageVideo API - an API that does do all the video work on the GPU. This API was introduced in Flash 10.2, which came out after this article from 2010.

jrockway 14y ago

the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem Really? The Internet at large is able to unanimously declare things, and one of those things is that it appreciates Flash's solution to the video problem? Tell me more...

vacri 14y ago
To be fair, when the article was written in January 2010, the only widespread solution to video in browsers was flash.
- TazeTSchnitzel 14y ago
  Same with audio, which is why notification sounds on Facebook and such were always done with Flash (not now).
  - lazugod 14y ago
    Facebook has sounds?
    - TazeTSchnitzel 14y ago
      Maybe you're using noscript. If you receive a chat message and you're in another tab, a sound plays.
- jrockway 14y ago
  Wasn't Netflix Silverlight then?
  - vacri 14y ago
    As an Australian, Netflix was never part of the web that I could see :)

Liongadev 14y ago

While watching streams of Olympia in both HTML5 (on Chrome) and Flash i found HTML5 to take nearly twice as much CPU then Flash. Might be due to the implementation of the sites (german broadcasters ard & zdf) but flash works pretty awesome for me.

badboy 14y ago
Besides that it's a shame that the option to just get the stream running in VLC is such a hassle (with VLC I get much lower CPU usage + the ability to freely move and resize the video and place it somewhere on my second screen, even fullscreen)

zacharyvoase 14y ago

Wow. A tip to anyone reading this: if you ever find yourself explaining away poor—nay, pathological—performance with anything other than ‘we made some mistakes in the past and we’re doing what we can to correct them’, please take a good hard look at yourself and consider exiting the software industry. That’s all I have to say. Edit: that wasn’t all I had to say. > What about Mac? I’m not sure but my Mac colleagues have mentioned something about Apple not making their hardware decoding APIs available to applications Well why didn’t you just go ahead and ask them before writing a blog post on company letterhead?! Is the corporate structure here so screwed up that you can’t actually do that, or are you just lazy?

pi18n 14y ago
Doubly so if your product crashes even half as often as Flash. That they cannot seem to code for stability doesn't give me any confidence in their ability to code for speed. I heard that they refused to work with Apple to improve its performance on Mac, and that was a core reason why they got excluded from the iOS platform -- searching for this I came up with an explanation directly from Apple: "Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash. We have been working with Adobe to fix these problems, but they have persisted for several years now. We don’t want to reduce the reliability and security of our iPhones, iPods and iPads by adding Flash."
- 0x0 14y ago
  You must have heard wrong, and it's the other way around. Apple did not open their video acceleration APIs until they released OS X 10.6.3 circla mid-2010. http://xbmc.org/davilla/2010/05/03/osx-gets-h-264-accelleration/ http://xbmc.org/davilla/2010/05/03/osx-gets-h-264-accellerat...
  - pi18n 14y ago
    If that's the major reason preventing them from being performant, then I misunderstood the reason for crashing and the reason for poor performance to be the same. Apple should not be complaining about CPU usage. That doesn't excuse Adobe from making buggy plugins.

illicium 14y ago

> Flash Player solves a different problem: It plays linear media files from start to finish while combining the video with a wide array of graphical and interactive elements (buttons, bitmaps, vector graphics, filters) And therein lies the problem with Flash video: dozens of video player implementations, each frustratingly different and user unfriendly than the next. People are building custom chrome on top of HTML5 video too, unfortunately, but at least it's standardized (and semantic) which allows users to consume web video with any interface they want.

jhack 14y ago

Wonder if Linux video drivers have added the ability to retrieve the decoded video data in the two and a half years since this blog post was made.

nicky0 14y ago

This is a blog post from January 2010. I think a lot of the details are out of date. (Mac hardware decoding support, for example.)

tluyben2 14y ago

I don't know if the explanation is accurate or just their own product defense, however (long) after playing a video (and not something else in flash), I usually go into top and kill the Chrome flash process manually as it's a) taking an insane amount of memory b) taking a lot of CPU for no reason (I'm not watching the video anymore and closed the page). Seems there is a ton of leakage going on after playing a video. So even though the story might to explain the actual playing of the video, it doesn't explain the molestation of my i7/8gb/ssd machine AFTER I closed (the only) Flash window. Not to mention the crashes.

lispm 14y ago

The message that I take is that some companies use Flash as the wrong tool to solve a problem which I'm not interested in. I want to see videos with minimum overhead. I don't want to be bothered with ads, poor performance, poor user interfaces, poor security, ... all which I get through Flash.

lmm 14y ago
Given the options available to them (roughly: expensive real video platform, expensive quicktime video platform, cheap content creation platform (in the form of flash) that can do video), those companies (and let's be honest, there's really only one that matters, youtube) made a perfectly reasonable decision. You might moan about flash interfaces, but unless you're willing to pay more (or, equivalently, view more ads) for a real, performant video interface (which was a genuine option for a while - e.g. the now-defunct stage6), you can't blame them for making that choice. (I feel it's a real failure of vision in the open-source community that there wasn't a free theora (say) plugin available for all browsers, and server-side video hosting platform, when youtube was getting started.)
- lispm 14y ago
  h264 has been a standard for a long time and performs very well on current hardware (many machines have hardware acceleration for it). Google has wasted terawatts of electricity due to flash.
  - lmm 14y ago
    Again, if users cared enough they would vote with their feet. It seems like no-hassle, low-ads video playback is worth more to us than playing it more efficiently.

0x0 14y ago

This is an extremely outdated article which was posted before they added "StageVideo", an API that fixes this problem 100%. http://www.adobe.com/devnet/flashplayer/stagevideo.html http://www.adobe.com/devnet/flashplayer/stagevideo.html

jessriedel 14y ago
So is this evidence that the blog author was full of it when he wrote this?
- 0x0 14y ago
  Not really, StageVideo has a bunch of limitations in the compositing pipeline, but that doesn't really matter if all you want to do is build a 2D movie player. Anyways, I guess Adobe changed their mind later, and decided they wanted to solve both of the "different problems" mentioned in the blog post. (Remember, Flash used to be all about vector animations. Sometimes it'd seem like the ubiquitous flash-as-a-video-player thing happened almost by accident; it turns out the video support (a relatively small feature in the whole flash featureset) was good enough to bring "universal" video to the web.)

rdsubhas 14y ago

tl;dr - we accidentally a whole software video decoder to show some circles and rectangles at the bottom. and so that you can haz internet when playing video.

alvarosm 14y ago

Ridiculous. YUV->RGB conversion + drawing a couple of things is negligibly cheap even if done in CPU. What a pathetic excuse.

acdha 14y ago
Not to mention that they were flat out lying when they claimed other players don't do this. Anyone who remembers when Mac devs were hand-tuning AltiVec implementations of YUV-RGB routines knows that Adobe was just choosing not to spend the money.
greyfade 14y ago
Yeah, that's pretty dubious. I was under the impression that all video players do that anyway - that video is encoded in the YUV color space and always converted to RGB on playback. To blame that for their poor performance suggests that their YUV-RGB conversion code is among the slowest in the industry.

its_so_on 14y ago

If only there were a way to let a plugin give VLC a URL to open and overlay over a specific part of your screen until you close the tab. Nah, that's crazy talk. Who would want to watch a high-quality hardware-accelerated local video that gets deleted when you close the tab, from the familiar interface of a web browser? Crazy talk, crazy talk.

inportb 14y ago
There is. http://www.videolan.org/doc/play-howto/en/ch04.html#id591206 http://www.videolan.org/doc/play-howto/en/ch04.html#id591206 Then again, I don't use it because I couldn't get any seek UI to appear.

brudgers 14y ago

"The Flash Player solves a different problem than your favorite video player." That problem is largely tracking user behavior across the internet.

ThisIsADogHello 14y ago
Don't forget that Flash also solves the problem of denying access to your selected media until unskippable ads have played, or overlaying said ads on top of the media you're trying to watch.

lincolnwebs 14y ago

The condescension in that post is galling. "Try to keep up here." "Again, say it with me:" "Please tell me you know people who aren’t as tech savvy as you. If you don’t, get to know some in order to maintain proper perspective." A lesson in how to not communicate a technical idea.

rocky1138 14y ago

This comment on the blog itself perfectly sums up my feeling about the article and its content, so I'll quote it here: "Again, say it with me… I’ve been reading this blog since the beginning. I always thought you were a little condescending, but you seemed knowledgeable enough and I thought you were doing good work.However, you can say this with me: Eat shit."