8 ms·
Why the HTML5 ‘Video’ Element Is Effectively Unusable
- tolmasky 17y agoThis is a problem that I find to be systemic in the HTML specs. Even in a world where every browser was somehow 100% compliant with the spec (never going to happen), we web developers would still be hitting our heads against the wall with browser inconsistencies because the spec chooses to make important behavior optional or subject to the browser-vendor's unique implementation. Another example of this is HTML5 drag and drop, where things like what exactly starts a drag is left completely up to the browser, resulting in major differences in behavior between browsers, to the point of almost being unusable.
- DarkShikari 17y agoAnother flaw in the spec is that it provides no means to specify client buffering behavior: you cannot tell the client how many seconds to buffer before playback. This completely breaks the buffering model inherent in modern video compression systems and basically only works properly at all if the users' connection is far higher than the bitrate of the video. Until HTML5 has basic feature parity with Flash's video capabilities, it will probably not be used very widely. This is of course ignoring the fact that Firefox doesn't support a video format that anyone actually uses, and the fact that IE (still unfortunately the most popular browser) doesn't support anything at all. The sad reality is that a single system often has the advantage over a group of conflicting, incompatible systems; even if the single system is bad, one at least only has to compensate for its weaknesses, and not for everything else's.
- deleted 17y ago[deleted]
- tumult 17y agoYou can actually control the buffering/playback with JavaScript. There are various ways to do this; one is with the TimeRanges object http://www.whatwg.org/specs/web-apps/current-work/#normalized-timeranges-object http://www.whatwg.org/specs/web-apps/current-work/#normalize... which will give you a representation of available time regions in the video resource. You can use this to determine when you want the client to begin playback. Your code can receive this after registering for an event and seeking the video. Another way is with the readyState of the media element. In particular you want to look for numeric value 4, HAVE_ENOUGH_DATA. From the WHATWG: All the conditions described for the HAVE_FUTURE_DATA state are met, and, in addition, the user agent estimates that data is being fetched at a rate where the current playback position, if it were to advance at the rate given by the defaultPlaybackRate attribute, would not overtake the available data before playback reaches the end of the media resource.
- DarkShikari 17y agoYou can actually control the buffering/playback with JavaScript. There are various ways to do this; one is with the TimeRanges object http://www.whatwg.org/specs/web-apps/current-work/#normalize.. http://www.whatwg.org/specs/web-apps/current-work/#normalize.... which will give you a representation of available time regions in the video resource. I could understand this in mp4, which has a global index giving locations of all frames in the file, but how would this be possible in ogg, which has no global index? Without a global index, you can't know anything about time regions that you haven't yet downloaded.
- tumult 17y agoThen it probably doesn't work with ogg :) I personally think it's a huge mistake for Firefox to support only ogg. It's a bad time for fragmentation in the push for open video on the web. It should still be possible to get a rough estimate for when to start playback. Good enough for buffered autoplay.
- pyre 17y agoThe problem being patents/licensing. mp4 is patented up the 'wazoo' and has onerous licensing terms.
- windsurfer 17y agoHow can Firefox distribute the H.264 codecs without infringing on patents?
- DarkShikari 17y agoHow do Linux distributions distribute H.264 and AAC decoders? Debian does, Ubuntu does, Fedora does, basically everything in the world except GNUsense does--yet they do just fine with themselves. The best solution would be to simply use system decoders, like Safari does. On Windows, this would be Directshow, on Linux this would be Gstreamer, and on OS X this would be Quicktime. The browser should have ABSOLUTELY NOTHING to do with video and audio decoding.
- est 17y agoWell, it won't be long till whatwg find another usability issue with `video` and stuff, and expanding HTML5 to a full featured bloated media player stack.
- aw3c2 17y agoYour constant Theora bashing is so wrong. H264 is patented, patents are bad, Theora seems the best alternative. Get over it.
- DarkShikari 17y agoWhen did I bash Theora? There's a lot wrong with Theora, but it's irrelevant to the discussion: what matters is what people actually use. If today it came up that there was a big patent on JPEG, would you propose that the entire web stop using JPEG tomorrow? No matter how good or bad Theora is, the internet will not simply drop the existing standard formats and switch. It doesn't work that way. This is also why browsers should use system decoders: it would make it easier to use a different video or audio format, since one could use anything that people had decoders installed for. This would stop the internet from being locked into a single format.
- aw3c2 17y agoI get the impression that you constantly do, but I might be wrong. System decoders (while being a sensible idea) actually reminds me of the web before Flash when people simply embedded wmv, mov etc. Why did one stop doing that again? Because of too many incompatibilities. And that's where you would end again. A standard format is the best solution, it is a shame the W3C was not that bold.
- ZeroGravitas 17y agoIt's a bit disingenuous to claim to promote choice when you're fully aware that the "choice" would be h.264 and that network effects would then render this a de-facto standard.
- jdowdell 17y agoSystem-level video decoders were used in Macromedia Director during multimedia and early Internet days, but developers found it didn't work very well... you had to choose an architecture(s), then had to test for its presence and do version-detection, then handle the installation/support process for audience members who needed it. "Use the system's video" sounds attractive, but just pushes the problem down the stack. Your mileage may differ, but I've seen people try it before....
- wmf 17y agoyou cannot tell the client how many seconds to buffer before playback. This completely breaks the buffering model inherent in modern video compression systems My understanding is that HTML5 browsers will attempt to buffer as much as possible; is there any reason you'd want less buffering?
- DenisM 17y agoSo that playback starts quicker?
- kierank 17y agoWhat do you mean by "as much as possible"?
- TeHCrAzY 17y agoWhen you don't want the video to start, when you only want to buffer the minimum required for smooth playback, to avoid wasted bandwidth when the user inevitablly closes the window (see youtube, they do this now), a myriad of reasons. Flexability allows inovation.
- warfangle 17y agoWhy does the browser itself not implement its own buffering algorithm? I do not quite understand why this onus should be placed on the developer responsible for the video imbedding functionality of the hosted datum. Certainly allow the external developer the flexibility to override this functionality (easily enough done via language features of JavaScript). But why force all developers to implement competing but similar algorithms? (by example and anecdotal personal observations: netflix > hulu > CBS/tv.com, in terms of how buffering algos deal with potentially high but intermittent bandwidth [1]). If this were abstracted to the browser level (discounting the fact that these long-format providers will likely never move away from their current flash/silverlight implementations), my choice of online tv watching would no longer be dependant on the competence of the content-providers buffering algorithm: currently, despite the fact that I enjoy several shows provided by CBS.com, I will not watch them online due to their shoddy buffering algos. I will not watch them via antenna, either (I watch on my own schedule, thanks). Of course, as stated, these providers are hopelessly locked to their delivery platform due to DRM and contractual obligations. Ergo, most of this point is moot when it comes to long-format TV-oriented content distribution. [1] in my experience, CBS does not buffer enough before playing, and does not buffer during the pause state. It also does not indicate how 'full' the buffer is during pause state, like hulu does. Netflix is, in my opinion, the best on this as it indicates on the seekbar the buffer level, and adjusts bitrate based on perceived bandwidth. When your bandwidth drops below the perceived bandwidth, it gives you an accurate status indication as to buffering activity. CBS, contrarily, simply buffers to the original perceived value and resumes play. This is problematic if you begin with high bandwidth but midway through stream something happens to negatively effect your bandwidth. If this situation occurs, you're likely to encounter a slideshow: video resumes for 5< seconds, whereupon it buffers once more. This results in >2s stutters every 3-10 seconds. Because CBS does not buffer during pause states, this renders the video unwatchable except during time periods when my bandwidth is constant: living on a highly variable cable node makes this time period nigh impossible to predict. Therefore I do not watch CBS shows - antenna or streaming - at all.
- ynniv 17y agoIts unusable because of auto-buffering? A simple onclick DOM replace will fix this (as the article mentions). I get that we want better tuning parameters, but really, the video tag is such an improvement over a flash player or embed tag that we're looking the gift horse in the mouth.
- pkulak 17y agoThat is a problem, but it seems like it would be very easy to move the image - click for video behavior into a pretty small JS library. Writing something like this probably wouldn't be all that terrible: <script type="text/javascript"> $(function() { $('#video_1').to_video('myvideo.mp4', 'myvideo.ogg'); }); </script> <img class="really_a_video" id="video_1" src="mythumb.jpg" /> I'd still rather do that than deal with Flash.
- bphogan 17y agoBut see that's kinda stinky cos it's not semantic. You're not embedding the video element in the document so it can describe the content. It would be really cool if the js somehow could remove the video element or something and add it back in... but I dunno how possible that is.
- jiaaro 17y agoHere's an idea: Why not just add an onclick that puts in the `<source>` tags? that way you still have a `<video>` tag but it has nothing to preload until you click it note: I havn't tested this approach Edit: Ok so I tried that in firefox and it doesn't work. But What if you make a video with 1 frame and a resolution of 1x1 and make that the source. Then you can replace that with onclick? (this one is more work than I'm willing to do out of curiosity, maybe someone else will try it?)
- dasil003 17y agoThe semantic part isn't even what really bothers me. I'm more concerned about requiring javascript for something that shouldn't need it.
- SecurityMatters 17y agoThe author demonstrated why Firefox, et al, made the correct choice on his example page. I visited his page using Firefox and the video would not play. My guess is that you need javascript turned on to play the videos on his site. I almost never enable javascript, and certainly would not do so just to watch a little video. It sounds like Firefox and others are trying to make the web pages more universally work.
- IgorPartola 17y agoI am sorry but that is insane. Today's web relies heavily on JavaScript. I recommend you get yourself a clean browser (maybe on a virtual machine), if you are paranoid about security (SecurityMatters huh?), and try using Google Maps, Google Calendar, et al.
- aw3c2 17y ago"Today's web" works just fine without Javascript if you browse around. One can easily enabled it per-site in good browsers for the sites where you actually need it. It has the nice benefit that a lot of distracting annoyances (just to mention snap.com, most advertising, those pesky "2000 social web icons popup" footers) simply disappear. If you want '"today's web"' with all its glittery blingbling and distractions, then go ahead. But don't fall for the illusion that you would miss anything by browsing with Javascript off by default.
- youngian 17y agoTurning off Javascript is not insane. JS opens you up to a wide range of possible security problems. Also, you can think of it ideologically - websites should still work with JS turned off. Building an accessible, semantic, well-formed website means building a website that works without Javascript. The only exceptions are Google Maps or anything equally as rich. And I do mean equally as rich.
- rimantas 17y agoYou know, times has changed recently. Now we have not only web sites, but the web apps too. And there different rules apply.
- kingkongreveng_ 17y agoJust post a normal ass link to a regular video file. Let the user's file type preferences and video player take over. Do the same for PDFs and audio. Why are people going through conniptions trying to get a markup reader to be a video player?
- bphogan 17y agoThey're not going through conniptions. A markup reader already can display images with the <img> tag, and video is to the web what images were in 1995. However, the spec has an easy fallback - anything inside the video tag will be displayed on the older browsers. SO, as you said, that's where a link to download the video could be. However, that's not what content producers want. They don't want you to be able to directly store the video on your machine, that's why they obscure it behind ridiculously obfuscated Flash players, making you go through all of these conniptions. That way you can't (easily) get it on your portable player or your home theater computer.
- generalk 17y agoHow does the title call out the video element as "effectively unusable" when in the end Gruber actually crafts a page that works with the video element? Sure, no client respects the autobuffer attribute, and there's no way to use a poster image without resorting to javascript, and you have to encode the video two different ways. But it still works. It works even if you stick to one format (you're just excluding clients) or don't play the javascript-dom-switchout game (the video autobuffers, you don't get a poster image at all). Better title: HTML5 Video Element Has Glaring Issues, but Usable.
- pohl 17y agoIf you look up "effectively" in the dictionary, it means "in such a manner as to achieve a desired result". The author highlights the desired result of requiring the user to click on the poster before buffering is allowed to begin. He currently cannot achieve the result he desires. The title makes sense to me.
- latortuga 17y agoAs a result of his rather drawn out explanation, however, it appears as though he can achieve the result he desires, just not as simply as he would like and in a manner he'd rather make fun of than use. I understand complaining that about the lack of a feature that everyone seems to agree is decent and desirable, not to mention spelled out in the specification but it is not relevant to that specific discussion to complain that "you can't do X" when you can in fact do X.
- pohl 17y agoI don't see anywhere in the article or in the title where he says that you can't do X. One needs to, again, sweep the author's qualifier "effectively" under the rug in order to construct that straw-man. The title says that you can't effectively do X using Y. I don't see how the fact that one can do X by instead using A and B in combination with Y invalidates that statement, especially with all the hype built up around Y as the savior of all from the evils of F. It doesn't feel right to drop carefully-chosen words in order to wage a critique.
- briansmith 17y ago"As for why I refuse to embed Flash, let me put it this way. I use and highly recommend ClickToFlash, which blocks all Flash content by default. Why would I publish content using a technology that I personally block by default?" Soon you will be blocking <audio> and <video> by default too, for the same reasons.
- tdm911 17y agoIt's not the video and audio aspects of flash he is concerned about, it's the automatic loading and playing of content with flash that is annoying. Audio and video that doesn't load until you ask for it is perfectly acceptable.
- bcl 17y agoFlashBlock handles that nicely in FF. Nothing is loaded until you click on it.
- dagw 17y agoThe auto play isn't a requirement for using flash. What makes you think that the people doing auto loading and playing with flash won't do it with HTML5 tags? People can write annoying crap on any platform.
- ugh 17y agoIs it? I block flash because on OS X Flash’s performance is ridiculous. I don’t care so much about auto play (the sites I visit are normally polite).
- newhouseb 17y agoSlightly related: the HTML5 'Audio' element is also effectively unusable. I was building choreographi.es (http://choreographi.es http://choreographi.es) a while back and was using the JPlayer plugin for JQuery, which attempts to use HTML5 audio and then falls back on Flash for platforms which don't support it. The gist of my app was that I needed to match movements _exactly_ to specific offsets in the music. Between Safari and Firefox, Firefox was dead on, while Safari randomly appeared to skew as it started playing from different points in the music, just enough to be significant. I'd imagine it was some bug in counting sampling rate. In the end I had to crack open the JPlayer code and force Flash all the time. TL;DR Moral of the story? It's hard to expect implementation to be perfect for a not-yet-finished standard, but until then, someone (you/me/us?) needs to make the equivalent of a community quirksmode combined with automatic bug submission to the relevant engine so we can get on with our lives.
- cpearce 17y agoUnfortunately John is entirely wrong, about Firefox at least. Firefox does not buffer the entire media when autobuffer is not present. I don't know how he came to that conclusion, but just check out this page as an example: http://pearce.org.nz/video/bunny-poster.html http://pearce.org.nz/video/bunny-poster.html Note that the progress bar on the controls does not turn to a lighter shade of gray until you press play - this means Firefox isn't buffering data unless you press play. John's blog post is wrong.
- latortuga 17y agoYou may be correct for Firefox but on the latest Chrome beta, my progress bar looks blue both before and after I press the play button - there is no indication to me as to whether it is buffering or waiting for input.
- telemachos 17y agoI'm on yesterday's Chromium build on a Mac, and here was my test. Go to the page, don't click, wait, wait some more, keep waiting, wait, some more wait. Then, try to drag the video to the half-way point and start play from there. Result? The video jumps back to the beginning of the play cycle. Thus, I deduce, it ain't loading. Firefox 3.5 here, no autobuffer. Safari does as Gruber says and starts loading immediately.
- theschwa 17y agoHere's an article from Mozilla going over this http://hacks.mozilla.org/2009/12/autobuffering-video-in-firefox/ http://hacks.mozilla.org/2009/12/autobuffering-video-in-fire...
- sgk284 17y agoThe argument seems kind of silly. In 1993 when people started using images in their content, I'm sure people were making similar points. "My browser shouldn't download an image until I tell it to, it uses my bandwidth!" or "Don't download the image until I click it." And the web would be a different place if things were kept that way. Now I'm not saying that the HTML5 video spec is perfect, but let me just provide a src and call it a day. Everything else should be out of my control.
- po 17y agoYou would change your mind in a hurry when your bandwidth bill comes in, or you run over your caps. That's basically what Gruber is saying: it's just not economically possible for people to embed video this way.
- fhars 17y agoThat is why at the time all browsers had an easily accessible configuration option "don't dowload images unless requested".