5 ms·

Be your own video hoster with Docker and Nginx

118 points by betablocker 11y ago

unnikked 11y ago
Awesome!
smegel 11y ago
Does docker run on FreeBSD? Because it is crazy to run NGINX on anything other than FreeBSD because of Linux's appalling bad async IO support. For all I know Windows might be a better choice.
- corobo 11y ago
  Out of interest not challenge, do you have any benchmarks for this at all? I'd be interested in seeing the end results
  - smegel 11y ago
    Well when NGINX introduced thread-pools on Linux to handle async tasks (mainly waiting for block device IO) I think the speed-up was around 8x. Thread-pools make NGINX on Linux tolerable, but it just works flawlessly and beautifully on FreeBSD.
    - corobo 11y ago
      Hey that's not a benchmark! :P
      - smegel 11y ago
        https://www.nginx.com/blog/thread-pools-boost-performance-9x/ https://www.nginx.com/blog/thread-pools-boost-performance-9x... ...and it was 9x not 8x...
        kbutler 11y ago
        The 9x delta is from "a synthetic benchmark that simulates the worst mix of blocking and non-blocking operations". Expect real-world workloads to have significantly less performance impact. The particular issue is: "The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases. To solve this problem in particular, thread pools were introduced in NGINX 1.7.11" So, NGINX doesn't think it's crazy to deploy on linux. And no, Windows is not a better option. "only the select() connection processing method is currently used, so high performance and scalability should not be expected." http://nginx.org/en/docs/windows.html http://nginx.org/en/docs/windows.html
- davexunit 11y ago
  What a bold, unsubstantiated claim.
- skreuzer 11y ago
  Docker does run of FreeBSD but it heavily relies on ZFS and the 64bit Linux compatibility layer. https://wiki.freebsd.org/Docker https://wiki.freebsd.org/Docker
- rubiquity 11y ago
  Aside from the thousands of applications using Nginx on Linux with fantastic results. Yes, yes we know *BSD has a better networking stack than Linux. Now that you're done letting everyone know how smart you are, please sit down.
  - smegel 11y ago
    > Aside from the thousands of applications using Nginx on Linux with fantastic results. Citation needed. Name a big company using nginx on Linux to serve up lots of static content. > Yes, yes we know *BSD has a better networking stack than Linux. Who mentioned anything about a network stack? > Now that you're done letting everyone know how smart you are Well given some people like you clearly don't understand the point I was making I think justifies the comment in the first place.
    - nl 11y ago
      Citation needed. Name a big company using nginx on Linux to serve up lots of static content. Facebook[1], Reddit[2], Wordpress[3] [1] http://venturebeat.com/2012/06/18/nginx-the-web-server-tech-youve-never-heard-of-that-powers-netflix-facebook-wordpress-and-more/ http://venturebeat.com/2012/06/18/nginx-the-web-server-tech-... [2] https://news.ycombinator.com/item?id=8244868 https://news.ycombinator.com/item?id=8244868 [3] http://searchdns.netcraft.com/?host=wordpress.com&x=0&y=10 http://searchdns.netcraft.com/?host=wordpress.com&x=0&y=10
- nl 11y ago
  There are millions of Ngnix on Linux production implementations. And your advice is to use experimental[1] Docker support in FreeBSD, when Nginx would be running inside Docker on top of FreBSD's Linux compatibility layer. Finally - and most critically - Ngnix introduced thread pools to specifically work around a limitation of Linux's async IO implementation: Although Linux provides a kind of asynchronous interface for reading files, it has a couple of significant drawbacks. One of them is alignment requirements for file access and buffers, but NGINX handles that well. But the second problem is worse. The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases. To solve this problem in particular, thread pools were introduced in NGINX 1.7.11.[2]. [1] https://wiki.freebsd.org/Docker https://wiki.freebsd.org/Docker [2] https://www.nginx.com/blog/thread-pools-boost-performance-9x/ https://www.nginx.com/blog/thread-pools-boost-performance-9x...
- jkldotio 11y ago
  I heard a while back - not sure where, it could have been on Slashdot but it escapes me - that FreeBSD was dying. Looking into the statistics just now it seems that's the case. FreeBSD is basically dead on the desktop[0] and is dying, no longer even holding a single percent, in the server market.[1] In the High Performance Computing (HPC) market, Linux holds over 99.8% of the top supercomputers.[2] [0] https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=9&qpcustomb=0 https://www.netmarketshare.com/operating-system-market-share... [1] http://w3techs.com/technologies/details/os-freebsd/all/all http://w3techs.com/technologies/details/os-freebsd/all/all [2] http://www.top500.org/statistics/details/osfam/1 http://www.top500.org/statistics/details/osfam/1
Animats 11y ago
This isn't being your own video hoster. This is the input side for feeding the live streaming feature of YouTube, etc. It's not a playout or distribution server, where viewers get their video direct from your server. That takes more bandwidth than you can pump through a Raspberry Pi or up most DSL lines. Also, most ISPs don't like you running a server, and streaming video servers are rather noticeable. Being your own video hoster usually means renting a server in a data center with serious bandwidth, or buying streaming service from some cloud provider.[1][2] [1] http://www.streamingvideoprovider.com/prices.html http://www.streamingvideoprovider.com/prices.html [2] https://aws.amazon.com/cloudfront/streaming/ https://aws.amazon.com/cloudfront/streaming/
- toomuchtodo 11y ago
  True, but you can use this to pipe your video source to a CDN for wide distribution. EDIT: If you're a CDN, it'd behoove you to extend this project to allow for one-click endpoint configuration. Just add API key/secret.
- dogma1138 11y ago
  I'm streaming from my home plexn box over my internet line even at 1080p the limit is more often than not the internet I have on my portable device at the moment. Within Europe 1080p is mostly manageable sub 720 is doable even in Asia. In have 40ish mbit upload capacity. Never had complaints from my ISP.
  - frik 11y ago
    40Mbit upload is common?
    - dogma1138 11y ago
      On residential fttc connections these days I would assume so.
      - Jamesbeam 11y ago
m00dy 11y ago
Hej, What is the revenue model ?
- toomuchtodo 11y ago
  Perhaps its just a cool project not trying to make money?
deleted 11y ago
[deleted]
bambax 11y ago
There's a typo on those links in the FAQ (restreamer is repeated): https://datarhei.github.io/restreamer/restreamer/wiki/dynamicdns_en.html https://datarhei.github.io/restreamer/restreamer/wiki/dynami... should be https://datarhei.github.io/restreamer/wiki/dynamicdns_en.html https://datarhei.github.io/restreamer/wiki/dynamicdns_en.htm... and https://datarhei.github.io/restreamer/restreamer/wiki/guide-buy-hardware.html https://datarhei.github.io/restreamer/restreamer/wiki/guide-... should be https://datarhei.github.io/restreamer/wiki/guide-buy-hardware.html https://datarhei.github.io/restreamer/wiki/guide-buy-hardwar... (but on that page, all links point to a 404).
dschiptsov 11y ago
Why should one put anything under Docker for production?
- dschiptsov 11y ago
  Perhaps, chrooted environment is required for effeciently serving static files, I don't know. And of all apps - nginx, which has been designed to be self-contained (process-isolation is good enough) and has almost zero external dependencies (openssl and pcre, which are parts of any modern OS). I could understand the idea to "containerize" that pile of Java crap with its 100500 maven dependencies just in order not to touch it again, but nginx... BTW, does Docker's "port mapping", which is, of course, has zero overhead, is what one needs in front of nginx in production?
  - kevan 11y ago
    Nginx is only one piece of the puzzle for this app, check out the Dockerfile[1]. [1] https://github.com/datarhei/restreamer/blob/master/Dockerfile https://github.com/datarhei/restreamer/blob/master/Dockerfil...
sandGorgon 11y ago
Does anyone know if the nginx+Netflix sendfile changes in Freebsd can be ported to Linux at some point?

118 points by betablocker 11y ago

Awesome!

Does docker run on FreeBSD? Because it is crazy to run NGINX on anything other than FreeBSD because of Linux's appalling bad async IO support. For all I know Windows might be a better choice.

corobo 11y ago
Out of interest not challenge, do you have any benchmarks for this at all? I'd be interested in seeing the end results
- smegel 11y ago
  Well when NGINX introduced thread-pools on Linux to handle async tasks (mainly waiting for block device IO) I think the speed-up was around 8x. Thread-pools make NGINX on Linux tolerable, but it just works flawlessly and beautifully on FreeBSD.
  - corobo 11y ago
    Hey that's not a benchmark! :P
    - smegel 11y ago
      https://www.nginx.com/blog/thread-pools-boost-performance-9x/ https://www.nginx.com/blog/thread-pools-boost-performance-9x... ...and it was 9x not 8x...
      - kbutler 11y ago
        The 9x delta is from "a synthetic benchmark that simulates the worst mix of blocking and non-blocking operations". Expect real-world workloads to have significantly less performance impact. The particular issue is: "The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases. To solve this problem in particular, thread pools were introduced in NGINX 1.7.11" So, NGINX doesn't think it's crazy to deploy on linux. And no, Windows is not a better option. "only the select() connection processing method is currently used, so high performance and scalability should not be expected." http://nginx.org/en/docs/windows.html http://nginx.org/en/docs/windows.html
davexunit 11y ago
What a bold, unsubstantiated claim.
skreuzer 11y ago
Docker does run of FreeBSD but it heavily relies on ZFS and the 64bit Linux compatibility layer. https://wiki.freebsd.org/Docker https://wiki.freebsd.org/Docker
rubiquity 11y ago
Aside from the thousands of applications using Nginx on Linux with fantastic results. Yes, yes we know *BSD has a better networking stack than Linux. Now that you're done letting everyone know how smart you are, please sit down.
- smegel 11y ago
  > Aside from the thousands of applications using Nginx on Linux with fantastic results. Citation needed. Name a big company using nginx on Linux to serve up lots of static content. > Yes, yes we know *BSD has a better networking stack than Linux. Who mentioned anything about a network stack? > Now that you're done letting everyone know how smart you are Well given some people like you clearly don't understand the point I was making I think justifies the comment in the first place.
  - nl 11y ago
    Citation needed. Name a big company using nginx on Linux to serve up lots of static content. Facebook[1], Reddit[2], Wordpress[3] [1] http://venturebeat.com/2012/06/18/nginx-the-web-server-tech-youve-never-heard-of-that-powers-netflix-facebook-wordpress-and-more/ http://venturebeat.com/2012/06/18/nginx-the-web-server-tech-... [2] https://news.ycombinator.com/item?id=8244868 https://news.ycombinator.com/item?id=8244868 [3] http://searchdns.netcraft.com/?host=wordpress.com&x=0&y=10 http://searchdns.netcraft.com/?host=wordpress.com&x=0&y=10
nl 11y ago
There are millions of Ngnix on Linux production implementations. And your advice is to use experimental[1] Docker support in FreeBSD, when Nginx would be running inside Docker on top of FreBSD's Linux compatibility layer. Finally - and most critically - Ngnix introduced thread pools to specifically work around a limitation of Linux's async IO implementation: Although Linux provides a kind of asynchronous interface for reading files, it has a couple of significant drawbacks. One of them is alignment requirements for file access and buffers, but NGINX handles that well. But the second problem is worse. The asynchronous interface requires the O_DIRECT flag to be set on the file descriptor, which means that any access to the file will bypass the cache in memory and increase load on the hard disks. That definitely doesn’t make it optimal for many cases. To solve this problem in particular, thread pools were introduced in NGINX 1.7.11.[2]. [1] https://wiki.freebsd.org/Docker https://wiki.freebsd.org/Docker [2] https://www.nginx.com/blog/thread-pools-boost-performance-9x/ https://www.nginx.com/blog/thread-pools-boost-performance-9x...
jkldotio 11y ago
I heard a while back - not sure where, it could have been on Slashdot but it escapes me - that FreeBSD was dying. Looking into the statistics just now it seems that's the case. FreeBSD is basically dead on the desktop[0] and is dying, no longer even holding a single percent, in the server market.[1] In the High Performance Computing (HPC) market, Linux holds over 99.8% of the top supercomputers.[2] [0] https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=9&qpcustomb=0 https://www.netmarketshare.com/operating-system-market-share... [1] http://w3techs.com/technologies/details/os-freebsd/all/all http://w3techs.com/technologies/details/os-freebsd/all/all [2] http://www.top500.org/statistics/details/osfam/1 http://www.top500.org/statistics/details/osfam/1

Animats 11y ago

This isn't being your own video hoster. This is the input side for feeding the live streaming feature of YouTube, etc. It's not a playout or distribution server, where viewers get their video direct from your server. That takes more bandwidth than you can pump through a Raspberry Pi or up most DSL lines. Also, most ISPs don't like you running a server, and streaming video servers are rather noticeable. Being your own video hoster usually means renting a server in a data center with serious bandwidth, or buying streaming service from some cloud provider.[1][2] [1] http://www.streamingvideoprovider.com/prices.html http://www.streamingvideoprovider.com/prices.html [2] https://aws.amazon.com/cloudfront/streaming/ https://aws.amazon.com/cloudfront/streaming/

toomuchtodo 11y ago
True, but you can use this to pipe your video source to a CDN for wide distribution. EDIT: If you're a CDN, it'd behoove you to extend this project to allow for one-click endpoint configuration. Just add API key/secret.
dogma1138 11y ago
I'm streaming from my home plexn box over my internet line even at 1080p the limit is more often than not the internet I have on my portable device at the moment. Within Europe 1080p is mostly manageable sub 720 is doable even in Asia. In have 40ish mbit upload capacity. Never had complaints from my ISP.
- frik 11y ago
  40Mbit upload is common?
  - dogma1138 11y ago
    On residential fttc connections these days I would assume so.
    - Jamesbeam 11y ago

m00dy 11y ago

Hej, What is the revenue model ?

toomuchtodo 11y ago
Perhaps its just a cool project not trying to make money?

deleted 11y ago

[deleted]

bambax 11y ago

There's a typo on those links in the FAQ (restreamer is repeated): https://datarhei.github.io/restreamer/restreamer/wiki/dynamicdns_en.html https://datarhei.github.io/restreamer/restreamer/wiki/dynami... should be https://datarhei.github.io/restreamer/wiki/dynamicdns_en.html https://datarhei.github.io/restreamer/wiki/dynamicdns_en.htm... and https://datarhei.github.io/restreamer/restreamer/wiki/guide-buy-hardware.html https://datarhei.github.io/restreamer/restreamer/wiki/guide-... should be https://datarhei.github.io/restreamer/wiki/guide-buy-hardware.html https://datarhei.github.io/restreamer/wiki/guide-buy-hardwar... (but on that page, all links point to a 404).

dschiptsov 11y ago

Why should one put anything under Docker for production?

dschiptsov 11y ago
Perhaps, chrooted environment is required for effeciently serving static files, I don't know. And of all apps - nginx, which has been designed to be self-contained (process-isolation is good enough) and has almost zero external dependencies (openssl and pcre, which are parts of any modern OS). I could understand the idea to "containerize" that pile of Java crap with its 100500 maven dependencies just in order not to touch it again, but nginx... BTW, does Docker's "port mapping", which is, of course, has zero overhead, is what one needs in front of nginx in production?
- kevan 11y ago
  Nginx is only one piece of the puzzle for this app, check out the Dockerfile[1]. [1] https://github.com/datarhei/restreamer/blob/master/Dockerfile https://github.com/datarhei/restreamer/blob/master/Dockerfil...

sandGorgon 11y ago

Does anyone know if the nginx+Netflix sendfile changes in Freebsd can be ported to Linux at some point?