3 ms·

The "Why Use JSMPEG" section hits the nail on the head with respect to the needs of some projects I have worked on. Particularly, I once needed to stream video

by cs2818 10y ago

The "Why Use JSMPEG" section hits the nail on the head with respect to the needs of some projects I have worked on. Particularly, I once needed to stream video from a UAV to multiple tablets and phones on a local network to allow collaborative annotations. Since it was a disaster response system there was no depending on external services, which at the time put WebRTC out of the picture (all of the easy to use implementations required internet access to use existing signaling services). We ended up using MJPEG and then later a JavaScript implementation of a MPEG-1 decoder. This library certainly would have made my life a little easier at the time!

rich90usa 10y ago
> there was no depending on external services, which at the time put WebRTC out of the picture (all of the easy to use implementations required internet access to use existing signaling services) See: https://github.com/cjb/serverless-webrtc https://github.com/cjb/serverless-webrtc That's a good rough demo of how WebRTC connections can still be established with the ask/offers being conveyed out-of-band. To make it a little more friendly for tablets (and to accomplish before messages expire) I'd think QR codes would be a reasonable way of passing the data without depending on an external service. There could also be some extraneous information that can be stripped to save on the amount of data you pass between peers so that the QR code isn't excessively gross (see: https://webrtchacks.com/the-minimum-viable-sdp/ https://webrtchacks.com/the-minimum-viable-sdp/)
networked 10y ago
Interesting. Did you look into how using the JavaScript MPEG-1 decoder over MJPEG affected the tablets' battery life?
- cs2818 10y ago
  I don't think we ever did formal testing on this (external battery packs were plentiful), but I do recall some of the early versions of the MPEG-1 decoder causing tablets to heat up.
donatj 10y ago
MJPEG is so freaking simple [1] and almost no one knows about it. It's one of my favorite things to play around with and in my opinion way underutilized. I built "drawing board" a while ago using an old school image map streaming a motion jpeg. You can interact with the video without ANY JavaScript [2]. The browser stays where you are on posting because the server returns an HTTP 204 - also highly underutilized in my opinion. 1. https://github.com/donatj/mjpeg-php/blob/master/mjpeg.php https://github.com/donatj/mjpeg-php/blob/master/mjpeg.php 2. https://github.com/donatj/imgboard https://github.com/donatj/imgboard
- niftich 10y ago
  MJPEG in casual parlance is more of a "technique" than a format -- and multipart/x-mixed-replace is a very loose definition of even something as loosely defined as MJPEG. Out of video containers, QuickTime introduced "Motion-JPEG" [1] with each frame in the container being a JPEG, and there have been formal definitions of how to communicate "JPEG-compressed video" with network protocols, like in RTP [2]. Meanwhile mixed-replace was a Netscape trick [3] to do server push with HTTP that never became a real force but was reimplemented by almost every browser trying to stay Mozilla-compatible (except IE), and happily used by webcams for dead-simple videostreaming. [1] https://developer.apple.com/standards/qtff-2001.pdf https://developer.apple.com/standards/qtff-2001.pdf [2] https://tools.ietf.org/html/rfc2435 https://tools.ietf.org/html/rfc2435 [3] https://web.archive.org/web/19981203153836/fishcam.netscape.com/assist/net_sites/pushpull.html https://web.archive.org/web/19981203153836/fishcam.netscape....
gm-conspiracy 10y ago
Can you elaborate on the use-case of that UAV project? It sounds rather interesting.
- cs2818 10y ago
  Yes. The link below is an early summary of the project [1], but I'll summarize here too: We were examining how the different individuals involved in a disaster response could best communicate their needs to the pilot of the UAV without causing significant cognitive burden on the pilot. We explored using sketching, virtual spotlights, and audio communication. The UAV's video stream and sensor data was pushed out to client devices where users could apply annotations to the video stream that would be reflected across all devices. The pilot would see these annotations unless they toggled out of the collaborative mode. Hardware-wise, we used a little ARM SBC (ODROID-U3) hooked into the UAV's existing piloting system to serve everything either direct to the client devices or back to servers in our mobile response lab. I believe we used Nexus 7 tablets for the UAV pilot and iPads for responders, but responders could also use their own devices if they preferred. [1] http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1247&context=cseconfwork&sei-redir=1 http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=12...
colek42 10y ago
You can actually do this with H264 as well. https://github.com/colek42/streamingDemo https://github.com/colek42/streamingDemo

by cs2818 10y ago

rich90usa 10y ago

> there was no depending on external services, which at the time put WebRTC out of the picture (all of the easy to use implementations required internet access to use existing signaling services) See: https://github.com/cjb/serverless-webrtc https://github.com/cjb/serverless-webrtc That's a good rough demo of how WebRTC connections can still be established with the ask/offers being conveyed out-of-band. To make it a little more friendly for tablets (and to accomplish before messages expire) I'd think QR codes would be a reasonable way of passing the data without depending on an external service. There could also be some extraneous information that can be stripped to save on the amount of data you pass between peers so that the QR code isn't excessively gross (see: https://webrtchacks.com/the-minimum-viable-sdp/ https://webrtchacks.com/the-minimum-viable-sdp/)

networked 10y ago

Interesting. Did you look into how using the JavaScript MPEG-1 decoder over MJPEG affected the tablets' battery life?

cs2818 10y ago
I don't think we ever did formal testing on this (external battery packs were plentiful), but I do recall some of the early versions of the MPEG-1 decoder causing tablets to heat up.

donatj 10y ago

MJPEG is so freaking simple [1] and almost no one knows about it. It's one of my favorite things to play around with and in my opinion way underutilized. I built "drawing board" a while ago using an old school image map streaming a motion jpeg. You can interact with the video without ANY JavaScript [2]. The browser stays where you are on posting because the server returns an HTTP 204 - also highly underutilized in my opinion. 1. https://github.com/donatj/mjpeg-php/blob/master/mjpeg.php https://github.com/donatj/mjpeg-php/blob/master/mjpeg.php 2. https://github.com/donatj/imgboard https://github.com/donatj/imgboard

niftich 10y ago
MJPEG in casual parlance is more of a "technique" than a format -- and multipart/x-mixed-replace is a very loose definition of even something as loosely defined as MJPEG. Out of video containers, QuickTime introduced "Motion-JPEG" [1] with each frame in the container being a JPEG, and there have been formal definitions of how to communicate "JPEG-compressed video" with network protocols, like in RTP [2]. Meanwhile mixed-replace was a Netscape trick [3] to do server push with HTTP that never became a real force but was reimplemented by almost every browser trying to stay Mozilla-compatible (except IE), and happily used by webcams for dead-simple videostreaming. [1] https://developer.apple.com/standards/qtff-2001.pdf https://developer.apple.com/standards/qtff-2001.pdf [2] https://tools.ietf.org/html/rfc2435 https://tools.ietf.org/html/rfc2435 [3] https://web.archive.org/web/19981203153836/fishcam.netscape.com/assist/net_sites/pushpull.html https://web.archive.org/web/19981203153836/fishcam.netscape....

gm-conspiracy 10y ago

Can you elaborate on the use-case of that UAV project? It sounds rather interesting.

cs2818 10y ago
Yes. The link below is an early summary of the project [1], but I'll summarize here too: We were examining how the different individuals involved in a disaster response could best communicate their needs to the pilot of the UAV without causing significant cognitive burden on the pilot. We explored using sketching, virtual spotlights, and audio communication. The UAV's video stream and sensor data was pushed out to client devices where users could apply annotations to the video stream that would be reflected across all devices. The pilot would see these annotations unless they toggled out of the collaborative mode. Hardware-wise, we used a little ARM SBC (ODROID-U3) hooked into the UAV's existing piloting system to serve everything either direct to the client devices or back to servers in our mobile response lab. I believe we used Nexus 7 tablets for the UAV pilot and iPads for responders, but responders could also use their own devices if they preferred. [1] http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1247&context=cseconfwork&sei-redir=1 http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=12...

colek42 10y ago

You can actually do this with H264 as well. https://github.com/colek42/streamingDemo https://github.com/colek42/streamingDemo