6 ms·

HTML Live Video Streaming Via WebSockets

134 points by phoboslab 13y ago

0x006A 13y ago
why is that better than any webcam using multipart jpg?
Jhsto 13y ago
As a side note, the page completely hangs Chrome for iOS.
- shurcooL 13y ago
  It does.
- phoboslab 13y ago
  Chrome for iOS is not allowed to use the JavaScript JIT, hence decoding the MPG is extremely slow. Try Mobile Safari and complain to Apple :)
kzahel 13y ago
Very cool! I like the simple approach of foregoing all the fancy new tech and doing the simplest possible thing (though the quality/bitrate does suffer)
deleted 13y ago
[deleted]
- deleted 13y ago
  [deleted]
bashtoni 13y ago
The Raspberry Pi has hardware video encoding support via the GPU. I dug up this blog post about it: http://theiopage.blogspot.co.uk/2013/04/enabling-hardware-h264-encoding-with.html http://theiopage.blogspot.co.uk/2013/04/enabling-hardware-h2... Would be great to integrate this with the node app.
- newman314 13y ago
  I was just wondering the same thing. Is there some other reason why he would not use the GPU to do h264 encode/decode? It appears that ffmpeg does support it.
deleted 13y ago
[deleted]
zimbatm 13y ago
It's not very clear from the article why this solution is superior to using a x-mixed-replace/multipart stream that contains image/jpeg bodies. If you don't have any audio just stream your jpegs using the x-mixed-replace HTTP response type. All browsers support it (even IE6).
- phoboslab 13y ago
  Better quality and framerate with lower bandwidth usage. At peak, the example in the article served 250 clients with smooth 30fps video while using about 120mbit/s. I agree that this is a huge hack and just serving JPGs would be much easier, but this hack turned out to work pretty well.
- mnutt 13y ago
  Unfortunately iOS broke the functionality somewhere around 5.1; it was still broken in 6 and I haven't checked iOS 7 yet. Animated gifs have the same issue: MobileSafari waits until the connection is closed before rendering anything.
DougWebb 13y ago
I just implemented something similar. I have an IP webcam on my network, but I noticed that the built-in viewer uses 30% cpu time on my laptop when I view it in Chrome. (20% on Firefox, about the same on IE using an ActiveX version of the client.) The built-in viewer supports a 'cell phone' mode that sends a stream of jpegs, but in an awful way that uses even more cpu time. I decided to write my own wrapper around the url it uses to get a jpeg of the current camera view. This uses requestAnimationFrame, and lets me set the max framerate I want. Since I'm just watching my dog Mischa and she sleeps most of the time, I set it to 0.5fps which keeps my cpu usage around 1%. <div> <img id="mischacam" /> <br /> <span id="time"/> </div> <script type="text/javascript"> var camURL = "...."; var msPerFrame = 2000; var loading = false; function update(timestamp) { if (loading) return; loading = true; lastFrameTime = timestamp; $("#mischacam").attr("src", camURL + "/snapshot.cgi?_=" + Math.random()); $("#time").html(new Date().toString()); } window.requestAnimFrame = (function(){ return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || function( callback ){ window.setTimeout(callback, 1000 / 60); }; })(); var lastFrameTime; function animloop(timestamp){ if (lastFrameTime == null) lastFrameTime = timestamp; if (timestamp - lastFrameTime > msPerFrame) update(timestamp); requestAnimFrame(animloop); } $(document).ready(function () { $("#mischacam").on("load", function () { loading = false; }); animloop(0); }); </script>
- ch4ch4 13y ago
  $("#mischacam").attr("src", camURL + "/snapshot.cgi?_=" + Math.random()); Out of curiosity, why do you append a random number between 0 and 1 to the URL?
  - mannix 13y ago
    Probably to bust possible caching.
  - DougWebb 13y ago
    mannix is right; to get the browser to reload the image I have to change the url every time. You'd think in a well-designed webcam API the image would come back with appropriate headers to prevent caching, but I didn't even bother to check. The webcam software is not well-designed.
deleted 13y ago
[deleted]
devongovett 13y ago
Super cool. I just submitted a pull request to jsmpeg to move some of the RGB conversion stuff to the GPU using WebGL. Reduces CPU usage by 5-10%. https://github.com/phoboslab/jsmpeg/pull/1 https://github.com/phoboslab/jsmpeg/pull/1
hardwaresofton 13y ago
This looks awesome, but I'd love to hear some comparisons between this hack (it is definitely far simpler) and WebRTC or multipart JPG. Also, great hack, it seems so obvious, but is actually pretty great.
paraboul 13y ago
It's awkward that in 2013 we have to complexify design to this point : "Streaming video through WebSocket (thus over http upgrade) through a plain-tcp-to-websocket-proxy on top of...". It feels over-engineered and like TCP-Sockets are a revolution. I know, we can't easily expose plain-socket because of security but still... What's next? UDP for 2018? Socket-binding for 2022? (Nice work though)
- makomk 13y ago
  Well, traditionally this would be done using something like Flash and RTMP, but Flash is dead on mobile - Apple considered it too CPU and battery hungry and a poor user experience, so they banned it. Hence MPEG decoders implemented in JavaScript.
  - paraboul 13y ago
    I think you missed my point. I'm not criticising what phobolabs (who is very talented BTW) has done or what any end-developer is doing. It's just crazy how the standard is slow to come up with new basic APIs. I mean, you can make a P2P video streaming application since 1995 using straightforward architecture in a desktop App using less cpu, less middleware and less engineering.
shurcooL 13y ago
I just tried your instant webcam app, and omg it is so cool. I've been looking for something like that. Very impressive low latency. It's free too, how are you making money? Thanks for making it! (Now I just need something for streaming video between laptops, between laptop and mobile device. I find it very annoying how FaceTime doesn't let you call yourself from one device to another.)
drivebyacct2 13y ago
Uh.... you reinvented the MediaSource API. :) Hopefully someone will see this and repeat it. (Google already has a Javascript implementation of MPEG-DASH [I have an unobfuscated copy, I believe it's still public on one of the Chromecast receiver apps]. And very likely HLS. It is supported on Android, ChromeCast, and Desktop. It's coming to Firefox eventually I believe.)
adambom 13y ago
I wonder if you could just encode each frame as a gif and send it to the browser as part of an animated gif a la gifsockets (https://github.com/videlalvaro/gifsockets https://github.com/videlalvaro/gifsockets)?
- bochi 13y ago
  I did that some time ago: https://github.com/jbochi/gifstreaming https://github.com/jbochi/gifstreaming
IgorPartola 13y ago
Just released something that streams video like this using MJPEG: https://igorpartola.com/projects/hawkeye/ https://igorpartola.com/projects/hawkeye/. I thought about using WebSockets but MJPEG seemed to be reasonably well supported by all the browsers I cared about do I chose to use it instead.

134 points by phoboslab 13y ago

0x006A 13y ago

why is that better than any webcam using multipart jpg?

Jhsto 13y ago

As a side note, the page completely hangs Chrome for iOS.

shurcooL 13y ago
It does.
phoboslab 13y ago
Chrome for iOS is not allowed to use the JavaScript JIT, hence decoding the MPG is extremely slow. Try Mobile Safari and complain to Apple :)

kzahel 13y ago

Very cool! I like the simple approach of foregoing all the fancy new tech and doing the simplest possible thing (though the quality/bitrate does suffer)

deleted 13y ago

[deleted]

deleted 13y ago
[deleted]

bashtoni 13y ago

The Raspberry Pi has hardware video encoding support via the GPU. I dug up this blog post about it: http://theiopage.blogspot.co.uk/2013/04/enabling-hardware-h264-encoding-with.html http://theiopage.blogspot.co.uk/2013/04/enabling-hardware-h2... Would be great to integrate this with the node app.

newman314 13y ago
I was just wondering the same thing. Is there some other reason why he would not use the GPU to do h264 encode/decode? It appears that ffmpeg does support it.

deleted 13y ago

[deleted]

zimbatm 13y ago

It's not very clear from the article why this solution is superior to using a x-mixed-replace/multipart stream that contains image/jpeg bodies. If you don't have any audio just stream your jpegs using the x-mixed-replace HTTP response type. All browsers support it (even IE6).

phoboslab 13y ago
Better quality and framerate with lower bandwidth usage. At peak, the example in the article served 250 clients with smooth 30fps video while using about 120mbit/s. I agree that this is a huge hack and just serving JPGs would be much easier, but this hack turned out to work pretty well.
mnutt 13y ago
Unfortunately iOS broke the functionality somewhere around 5.1; it was still broken in 6 and I haven't checked iOS 7 yet. Animated gifs have the same issue: MobileSafari waits until the connection is closed before rendering anything.

DougWebb 13y ago

I just implemented something similar. I have an IP webcam on my network, but I noticed that the built-in viewer uses 30% cpu time on my laptop when I view it in Chrome. (20% on Firefox, about the same on IE using an ActiveX version of the client.) The built-in viewer supports a 'cell phone' mode that sends a stream of jpegs, but in an awful way that uses even more cpu time. I decided to write my own wrapper around the url it uses to get a jpeg of the current camera view. This uses requestAnimationFrame, and lets me set the max framerate I want. Since I'm just watching my dog Mischa and she sleeps most of the time, I set it to 0.5fps which keeps my cpu usage around 1%. <div> <img id="mischacam" /> <br /> <span id="time"/> </div> <script type="text/javascript"> var camURL = "...."; var msPerFrame = 2000; var loading = false; function update(timestamp) { if (loading) return; loading = true; lastFrameTime = timestamp; $("#mischacam").attr("src", camURL + "/snapshot.cgi?_=" + Math.random()); $("#time").html(new Date().toString()); } window.requestAnimFrame = (function(){ return window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimationFrame || function( callback ){ window.setTimeout(callback, 1000 / 60); }; })(); var lastFrameTime; function animloop(timestamp){ if (lastFrameTime == null) lastFrameTime = timestamp; if (timestamp - lastFrameTime > msPerFrame) update(timestamp); requestAnimFrame(animloop); } $(document).ready(function () { $("#mischacam").on("load", function () { loading = false; }); animloop(0); }); </script>

ch4ch4 13y ago
$("#mischacam").attr("src", camURL + "/snapshot.cgi?_=" + Math.random()); Out of curiosity, why do you append a random number between 0 and 1 to the URL?
- mannix 13y ago
  Probably to bust possible caching.
- DougWebb 13y ago
  mannix is right; to get the browser to reload the image I have to change the url every time. You'd think in a well-designed webcam API the image would come back with appropriate headers to prevent caching, but I didn't even bother to check. The webcam software is not well-designed.

deleted 13y ago

[deleted]

devongovett 13y ago

Super cool. I just submitted a pull request to jsmpeg to move some of the RGB conversion stuff to the GPU using WebGL. Reduces CPU usage by 5-10%. https://github.com/phoboslab/jsmpeg/pull/1 https://github.com/phoboslab/jsmpeg/pull/1

hardwaresofton 13y ago

This looks awesome, but I'd love to hear some comparisons between this hack (it is definitely far simpler) and WebRTC or multipart JPG. Also, great hack, it seems so obvious, but is actually pretty great.

paraboul 13y ago

It's awkward that in 2013 we have to complexify design to this point : "Streaming video through WebSocket (thus over http upgrade) through a plain-tcp-to-websocket-proxy on top of...". It feels over-engineered and like TCP-Sockets are a revolution. I know, we can't easily expose plain-socket because of security but still... What's next? UDP for 2018? Socket-binding for 2022? (Nice work though)

makomk 13y ago
Well, traditionally this would be done using something like Flash and RTMP, but Flash is dead on mobile - Apple considered it too CPU and battery hungry and a poor user experience, so they banned it. Hence MPEG decoders implemented in JavaScript.
- paraboul 13y ago
  I think you missed my point. I'm not criticising what phobolabs (who is very talented BTW) has done or what any end-developer is doing. It's just crazy how the standard is slow to come up with new basic APIs. I mean, you can make a P2P video streaming application since 1995 using straightforward architecture in a desktop App using less cpu, less middleware and less engineering.

shurcooL 13y ago

I just tried your instant webcam app, and omg it is so cool. I've been looking for something like that. Very impressive low latency. It's free too, how are you making money? Thanks for making it! (Now I just need something for streaming video between laptops, between laptop and mobile device. I find it very annoying how FaceTime doesn't let you call yourself from one device to another.)

drivebyacct2 13y ago

Uh.... you reinvented the MediaSource API. :) Hopefully someone will see this and repeat it. (Google already has a Javascript implementation of MPEG-DASH [I have an unobfuscated copy, I believe it's still public on one of the Chromecast receiver apps]. And very likely HLS. It is supported on Android, ChromeCast, and Desktop. It's coming to Firefox eventually I believe.)

adambom 13y ago

I wonder if you could just encode each frame as a gif and send it to the browser as part of an animated gif a la gifsockets (https://github.com/videlalvaro/gifsockets https://github.com/videlalvaro/gifsockets)?

bochi 13y ago
I did that some time ago: https://github.com/jbochi/gifstreaming https://github.com/jbochi/gifstreaming

IgorPartola 13y ago

Just released something that streams video like this using MJPEG: https://igorpartola.com/projects/hawkeye/ https://igorpartola.com/projects/hawkeye/. I thought about using WebSockets but MJPEG seemed to be reasonably well supported by all the browsers I cared about do I chose to use it instead.