5 ms·

Cool! Some thoughts from a former Twitch engineer: - Probably the hardest part of running these things is managing outbound bandwidth costs. You'll need to eit

Cool! Some thoughts from a former Twitch engineer: - Probably the hardest part of running these things is managing outbound bandwidth costs. You'll need to either limit inbound bitrate or transcode video down to a manageable rate, or else you'll quickly spend a lot of money on shipping 4k video for people. - Right now, your nginx hosts both do ingest and playback, if I understand it right. You might want to separate the two. It makes maintenance easier, and it lets you scale much better - right now, a single stream probably maxes out on viewership based on the CPU capacity of the single nginx host that is ingesting the stream, transcoding it, and delivering it. If you have multiple nginx hosts which could deliver the already-transcoded stream, you could scale much better. - Please don't keep using RTMP. RTMP is so stateful that it's pretty hard to manage, it doesn't have a spec, it doesn't have implementations on many devices, and its security is, uh, _weak_. Big players are forced to keep using it because telling their broadcasters to change is hard, but you don't have that problem. You might consider accepting an alternative modern protocol. - You'll almost certainly need admin and moderation tools soon. Expect lots of pirate streams, as well as some horrific content. You can't run a live streaming platform without admin tools. - Beware DDoS attacks. This setup looks very, very, very easy to take down, as-is...

toomuchtodo 6y ago
> Please don't keep using RTMP. ... You might consider accepting an alternative modern protocol. What would you recommend as an alternative modern protocol to RTMP for shipping video between a broadcast endpoint and transcoding/distribution infra?
- grogenaut 6y ago
  Spenczar might know better but SRT seems very interesting for modern protocols. RTMP is very old as a protocol.
  - tleb_ 6y ago
    > RTMP is very old as a protocol. Would you consider that this is, by itself, a reason not to use it?
  - iso1631 6y ago
    RTMP is very old as a protocol and pretty rubbish. But not because it's old. IP is old. UDP is old. RTP is old. Using TCP for live streaming isn't a good thing, same as using TCP for a VPN isn't good. SRT isn't great, it grew from a file transfer program (UDT). UDP file transfers like UDP and Signiant were popular 15 years ago when TCP window scaling and buffers were more miss than hit. Companies like signiant and filecatalyst were very big on them, and UDT was an attempt to standardise it. RIST is technically a far nicer protocol than SRT, but suffered massively from the talking-shop development. It came out two years late, and even then didn't have the proper libraries to just drop into your prosumer camera like SRT did. N.B. It's a shame as their file transfer management systems were good (control, monitoring, security, scheduling etc), but they were going full pelt on the "TCP is slow, UDP is fast" line - to the point that they deliberatly took standard linux distributions like ubuntu and redhat, and changed the default settings to disable tcp window scaling, so show a "side by side" comparison on their sales booths at events.
- spenczar5 6y ago
  Well, to be fair, there isn't a clear winner. Maybe SRT, but it doesn't have much platform support itself, and its spec is pretty strange. I get weird Adobe flashbacks about it, since it seems so heavily pushed by Haivision, and it's really made for a different use case - it's made for big broadcasters like ESPN and TV networks, not for web startups. The current motion seems to be towards hacking WebRTC to accommodate broadcast ingest. But WebRTC is really huge and complicated and you only want a tiny sliver of it for this use case. Pretty much all implementations are buggy because its such a gigantic kitchen-sink design. There's also RIST (for those who love working groups) and Zixi (for those who love closed source protocols?). I honestly think that something like HLS could do pretty well (broadcaster slices video into 1-second TS segments and just ships em with HTTP POST), but it's not something I ever saw anyone do. Definitely doesn't meet the TV networks' needs, but its really easy to scale and straightforward to secure, which is maybe more important.
  - Sean-Der 6y ago
    WebRTC really isn't that big. Google's implementation just is targeted at their needs. If you are doing it for ingest you just need * Basic SDP library, ICE-Lite, DTLS, SRTP If you ship RIST you pretty much end up with the same thing! If you go with WebRTC you have implementations in lots of languages as well (not just bindings) * Go, Python, Pure C/Embedded, Java, Rust, C++ Also WebRTC could let you do SVC or Simulcast. It would be a lot better for scaling if the uploader sent all the different quality feeds (and let congestion control ensure they don't oversend). I really feel that there has been a concentrated marketing effort that has hurt WebRTC on multiple fronts. * Google oversells what they have done. They bundled up a bunch of existing protocols. That doesn't get as my promotions/awards though. It also helps them control the standard. If they paint 'Google WebRTC === WebRTC' then they make a change and everyone else has to play ball. * WebRTC competitors want to paint it in a bad light so they can sell $x
    - spenczar5 6y ago
      I dunno, https://www.w3.org/TR/webrtc/ https://www.w3.org/TR/webrtc/ seems pretty huge to me. And then there are all the RFCs and notes you need to also read and understand, and then you realize that the real spec seems to be Google’s implementation... that at least was how things felt in 2018. They totally might have progressed since then. Yes, you can pick from the WebRTC buffet and build a good ingest protocol, but you need a lot of knowledge to do that - I would guess a tiny tiny number of people who read your comment know what DTLS or even STUN are. I also agree that there is a branding problem, but I think part of it is that we don’t have a good name for this WebRTC ingest pattern. It deserves a name and some attempt at standardization so tools and software can start building towards it without supporting everything in WebRTC. Because yeah, I think its the least-bad thing we have right now. I think you must be the same Sean-Der who works on the Go WebRTC library? Thanks a ton - that’s serious open source work.
- gizah 6y ago
  (disclaimer: I'm participating in the libRIST development) RIST! It's a well thought out protocol without technical debt (which SRT has), supports multicast (if you need it), null packet deletion, encryption via PSK or DTLS and multiplexing (multiplexing support in libRIST is not fully there yet, it's on the roadmap together with DTLS). Next iteration of the RIST protocol will support "backpressure" on your encoder, so if you have too much packet loss your encoding can scale it's video bitrate down, while together with null packet deletion you'll be able to keep your bitrate perfectly CBR.
  - stormbrew 6y ago
    This sounds interesting but until it's in ffmpeg and/or obs (being in ffmpeg would put it in obs more or less by default eventually now I think) that'll limit its practical usefulness for a lot of people. Is that planned/being pushed for?
- iso1631 6y ago
  RTP, if you need error correction use RIST simple profile. It's really simple and you can see exactly what's happening on the network.
nowahe 6y ago
Having worked with AWS IVS, which is using the same infrastructure design as Twitch (or the other way around), it only accepts RTMP (well RTMPS to be fair) as an input. To me it would seem that the industry is going the opposite way, and doubling down on RTMP. An RTMP connector is planned on the Chime SDK as well (a framework & infrastructure based on Chime to create your own custom meetings), to be able to stream the meeting directly to ingest service (like IVS/Elemental). On the AWS side at least, there doesn't seem to be a plan to migrate from RTMP, as it seems that services are being launched that only supports it. On a sidenote, the official AWS recommended way to stream a Chime SDK meeting to IVS, is to use this[0] docker container, acting as a bridge between WebRTC & RTMP. I find the hackiness of it amazing, as it's mostly just an X11 framebuffer, an instance of Firefox loading the webview for the meeting, and ffmpeg capturing the framebuffer and sending it as an RTMP stream to the configured endpoint. Also, AWS IVS as a streaming platform is really impressive. It's a fully managed service, and out of the box you get around 4-5s of latency between the source and the player, with an upper limit of 1080p60 @8.5Mbps. It will also downsample the stream to 720p, 480p, 360p & 160p. [0]: https://github.com/aws-samples/amazon-chime-meeting-broadcast-demo https://github.com/aws-samples/amazon-chime-meeting-broadcas...
- jiofih 6y ago
  Is 4-5s latency considered good? It’s an awful experience for video chat.
  - nowahe 6y ago
    Might have mixed up Chime & IVS a bit. To clarify, the 4-5s delay I'm talking about is for IVS, which is a live streaming solution, and doesn't have anything to do with video chat. It has a RTMP input on which you send a video stream, re-encodes it and then distribute it as an HLS stream. And getting sub 5s latency on an HLS stream is pretty fast. I brought up Chime and video chat mostly as an example on how AWS is pushing for RTMP to connect those services.
  - colde 6y ago
    For live streaming distribution to loads of viewers, outside of a video meeting scenario, then yeah. That is considered good. Plenty of commercial services with latency 5-10 times that.
sneak 6y ago
> Probably the hardest part of running these things is managing outbound bandwidth costs. This might be a good time to mention that Hetzner (where I host) has a lot of unmetered gigabit options. I'm not affiliated, just a happy (long term) customer. Sadly Germany has some laws that censor certain types of harmless content, so that's one downside, but that's not Hetzner's fault, just something to consider depending on what content you intend to host.
- dewey 6y ago
  > Sadly Germany has some laws that censor certain types of harmless content What is this harmless content that's censored?
  - sneak 6y ago
    Video game gore/violence is one example, though there are many others; German censorship law is relatively broadly worded. Games sold in Germany have to have their gore reduced (relative to versions legal in other countries) to be legal for publication in Germany. I'm not a lawyer, but my amateur assumption is that publication of video streams is subject to the same content rules as publication of game software.
- nielsole 6y ago
  You could use their Finnish dc. (no idea if Finnish laws are any better in this regard)
- chaz6 6y ago
  I can attest to Hetzner. I pay €35 per month for a dedicated server, and one of the things I run is a (non-exit) Tor relay, which consumes 3.5TiB per day on average.
supermatt 6y ago
Any specifics on DoS? What sort of strategies to prevent a malicious user pushing multiple streams to DoS encoders, etc?
ChrisMarshallNY 6y ago
I don’t like RT[M|S]P, but I have not seen an acceptable alternative for realtime streaming. The main reason I don’t like RTMP, is because it isn’t supported at the OS level in Apple systems (the platform I program for). HLS doesn’t come close to RTMP. It’s designed for movie streaming, and doesn’t support realtime streaming. I write video surveillance software, and the HLS lag is unacceptable.
zweizeichen 6y ago
Some great points! I built a project similar to this (think online lectures but it never went anywhere) in spring when the pandemic hit. I highly recommend building a platform like this if you have some spare time. It involves lots of interesting engineering/architecture challenges all over the stack. IMHO, from a technical viewpoint, it's mostly integration work, as all the really hard parts were already done by excellent 3rd party tools and libraries. You'll probably have some kind of RTMP bridge (nginx [1]), wrap ffmpeg for transcoding and play it back using something like Shaka player [2] where the segments are being served by some kind of caching reverse proxy. It took me a few weeks to get a prototype running inside a K8S cluster where oAuth-ed users could publish RTMPS streams which could be watched by other authed users via HLS in a somewhat scalable way. It was surprisingly easy to build this using Elixir/Phoenix LiveView. Some thoughts on your comment: > - Probably the hardest part of running these things is managing outbound bandwidth costs. This. As others noted, you may be getting around this by using something like Hetzner's/Scaleway's/OVH's offerings. However, I think they won't be too happy if you really use that much bandwidth all the time. You can probably forget using IaaS of one of the bigger cloud platforms unless you negotiated some special terms. > - You'll almost certainly need admin and moderation tools soon. That's one of the main reasons why I did offer a public instance. It's probably best to host a platform like this in a non-public context such as members of a single org. Just look at what happened to Firefox Send... > - Please don't keep using RTMP [...] and its security is, uh, _weak_. Yes it is a bit of a pain to work with but AFAIK you can wrap RTMP inside a TLS session which is supported by OBS. I think I just exposed a stunnel instance back then which forwarded it to a nginx instance which handled auth/auth and forwarding to the transcoding backend. This way you won't leak any streaming keys or video data. Please correct me if I'm wrong. If you have any additional pointers regarding RTMP security, I would be highly interested! Also, as others pointed out, SRT may be just around the corner. I think we're in for some interesting times as LL-DASH/CMAF and similar technologies are more widely supported. Additionally, there are interesting projects like [3] going the WebRTC route for delivering low latency streams. [1] https://github.com/arut/nginx-rtmp-module https://github.com/arut/nginx-rtmp-module [2] https://github.com/google/shaka-player https://github.com/google/shaka-player [3] https://www.ovenmediaengine.com https://www.ovenmediaengine.com
andrenotgiant 6y ago
FWIW, the OP repo uses a Terraform config to deploy the server on a 4GBRAM server on DigitalOcean.[1] Which costs $20/mon and has 4TB/mon bandwidth allowance.[2] To give a sense of what a 4TB bandwidth could cover: 320 people watching a 3hr stream at 1080p or 70mb per minute. [1] https://github.com/benwilber/boltstream/blob/master/terraform/terraform.tfvars https://github.com/benwilber/boltstream/blob/master/terrafor... [2] https://www.digitalocean.com/community/tools/bandwidth?active=%5B%7B%22slug%22%3A%22s-2vcpu-4gb%22%2C%22type%22%3A%22droplet%22%2C%22hours%22%3A744%2C%22consumption%22%3A12600%2C%22nodes%22%3A1%7D%5D&additional=0 https://www.digitalocean.com/community/tools/bandwidth?activ...
stormbrew 6y ago
"Please don't keep using RTMP." So, this has been talked about a bunch already, but I'm involved in a small streaming group that occasionally puts on (online, not in person) marathons and we've been looking for something to use that's better than rtmp for a while for ingest from participants and we really haven't found anything good. At least nothing that's viable to set up with open source software of your own. We just piloted SRT (having about 35 different streamers stream to us with it from OBS into srt-live-server, then into OBS with a media source), which has been mentioned a bunch here, and it's been a mixed bag and I don't know if we'll stick with it. The big pro is that it doesn't fall behind like the usual RTMP chain (OBS->nginx-rtmp-flv->OBS media source or web source with flvjs) tends to when connection quality degrades. I don't think this is necessarily an inherent characteristic of rtmp really, but the open software in this chain tends to just grow bigger and bigger buffers when the connection hitches, leading to progressively worse latency you have to 'fix' by resetting the stream. Twitch and other big ingesters seem to avoid this somehow, and I suspect ingesting directly into HLS or similar avoids it, but low latency HLS is still essentially unavailable in open source packaging so is not viable for our use. On the downside, though, SRT requires somewhat manual tweaking of latency and buffering parameters to work properly and our experience has been that, while the stream never falls behind, hitches outside the configured parameters produce a particularly ghastly kind of artifacting we've called "streakies" where entire segments of the feed will just turn weird bands of colours. It's possible this is actually a problem in the software, as the bits needed to put this together (particular srt-live-server) seem pretty immature yet. It seems like the future for this kind of use case is likely to wind up being webrtc, but as of now getting webrtc to play nice with OBS is kind of painful, usually just involving using external software that captures a popped out preview display from OBS. It's a little less than ideal. I think it'll be our next thing to try though, and obs.ninja looks kind of promising as a platform to help with doing it.

toomuchtodo 6y ago

> Please don't keep using RTMP. ... You might consider accepting an alternative modern protocol. What would you recommend as an alternative modern protocol to RTMP for shipping video between a broadcast endpoint and transcoding/distribution infra?

grogenaut 6y ago
Spenczar might know better but SRT seems very interesting for modern protocols. RTMP is very old as a protocol.
- tleb_ 6y ago
  > RTMP is very old as a protocol. Would you consider that this is, by itself, a reason not to use it?
- iso1631 6y ago
  RTMP is very old as a protocol and pretty rubbish. But not because it's old. IP is old. UDP is old. RTP is old. Using TCP for live streaming isn't a good thing, same as using TCP for a VPN isn't good. SRT isn't great, it grew from a file transfer program (UDT). UDP file transfers like UDP and Signiant were popular 15 years ago when TCP window scaling and buffers were more miss than hit. Companies like signiant and filecatalyst were very big on them, and UDT was an attempt to standardise it. RIST is technically a far nicer protocol than SRT, but suffered massively from the talking-shop development. It came out two years late, and even then didn't have the proper libraries to just drop into your prosumer camera like SRT did. N.B. It's a shame as their file transfer management systems were good (control, monitoring, security, scheduling etc), but they were going full pelt on the "TCP is slow, UDP is fast" line - to the point that they deliberatly took standard linux distributions like ubuntu and redhat, and changed the default settings to disable tcp window scaling, so show a "side by side" comparison on their sales booths at events.
spenczar5 6y ago
Well, to be fair, there isn't a clear winner. Maybe SRT, but it doesn't have much platform support itself, and its spec is pretty strange. I get weird Adobe flashbacks about it, since it seems so heavily pushed by Haivision, and it's really made for a different use case - it's made for big broadcasters like ESPN and TV networks, not for web startups. The current motion seems to be towards hacking WebRTC to accommodate broadcast ingest. But WebRTC is really huge and complicated and you only want a tiny sliver of it for this use case. Pretty much all implementations are buggy because its such a gigantic kitchen-sink design. There's also RIST (for those who love working groups) and Zixi (for those who love closed source protocols?). I honestly think that something like HLS could do pretty well (broadcaster slices video into 1-second TS segments and just ships em with HTTP POST), but it's not something I ever saw anyone do. Definitely doesn't meet the TV networks' needs, but its really easy to scale and straightforward to secure, which is maybe more important.
- Sean-Der 6y ago
  WebRTC really isn't that big. Google's implementation just is targeted at their needs. If you are doing it for ingest you just need * Basic SDP library, ICE-Lite, DTLS, SRTP If you ship RIST you pretty much end up with the same thing! If you go with WebRTC you have implementations in lots of languages as well (not just bindings) * Go, Python, Pure C/Embedded, Java, Rust, C++ Also WebRTC could let you do SVC or Simulcast. It would be a lot better for scaling if the uploader sent all the different quality feeds (and let congestion control ensure they don't oversend). I really feel that there has been a concentrated marketing effort that has hurt WebRTC on multiple fronts. * Google oversells what they have done. They bundled up a bunch of existing protocols. That doesn't get as my promotions/awards though. It also helps them control the standard. If they paint 'Google WebRTC === WebRTC' then they make a change and everyone else has to play ball. * WebRTC competitors want to paint it in a bad light so they can sell $x
  - spenczar5 6y ago
    I dunno, https://www.w3.org/TR/webrtc/ https://www.w3.org/TR/webrtc/ seems pretty huge to me. And then there are all the RFCs and notes you need to also read and understand, and then you realize that the real spec seems to be Google’s implementation... that at least was how things felt in 2018. They totally might have progressed since then. Yes, you can pick from the WebRTC buffet and build a good ingest protocol, but you need a lot of knowledge to do that - I would guess a tiny tiny number of people who read your comment know what DTLS or even STUN are. I also agree that there is a branding problem, but I think part of it is that we don’t have a good name for this WebRTC ingest pattern. It deserves a name and some attempt at standardization so tools and software can start building towards it without supporting everything in WebRTC. Because yeah, I think its the least-bad thing we have right now. I think you must be the same Sean-Der who works on the Go WebRTC library? Thanks a ton - that’s serious open source work.
gizah 6y ago
(disclaimer: I'm participating in the libRIST development) RIST! It's a well thought out protocol without technical debt (which SRT has), supports multicast (if you need it), null packet deletion, encryption via PSK or DTLS and multiplexing (multiplexing support in libRIST is not fully there yet, it's on the roadmap together with DTLS). Next iteration of the RIST protocol will support "backpressure" on your encoder, so if you have too much packet loss your encoding can scale it's video bitrate down, while together with null packet deletion you'll be able to keep your bitrate perfectly CBR.
- stormbrew 6y ago
  This sounds interesting but until it's in ffmpeg and/or obs (being in ffmpeg would put it in obs more or less by default eventually now I think) that'll limit its practical usefulness for a lot of people. Is that planned/being pushed for?
iso1631 6y ago
RTP, if you need error correction use RIST simple profile. It's really simple and you can see exactly what's happening on the network.

nowahe 6y ago

Having worked with AWS IVS, which is using the same infrastructure design as Twitch (or the other way around), it only accepts RTMP (well RTMPS to be fair) as an input. To me it would seem that the industry is going the opposite way, and doubling down on RTMP. An RTMP connector is planned on the Chime SDK as well (a framework & infrastructure based on Chime to create your own custom meetings), to be able to stream the meeting directly to ingest service (like IVS/Elemental). On the AWS side at least, there doesn't seem to be a plan to migrate from RTMP, as it seems that services are being launched that only supports it. On a sidenote, the official AWS recommended way to stream a Chime SDK meeting to IVS, is to use this[0] docker container, acting as a bridge between WebRTC & RTMP. I find the hackiness of it amazing, as it's mostly just an X11 framebuffer, an instance of Firefox loading the webview for the meeting, and ffmpeg capturing the framebuffer and sending it as an RTMP stream to the configured endpoint. Also, AWS IVS as a streaming platform is really impressive. It's a fully managed service, and out of the box you get around 4-5s of latency between the source and the player, with an upper limit of 1080p60 @8.5Mbps. It will also downsample the stream to 720p, 480p, 360p & 160p. [0]: https://github.com/aws-samples/amazon-chime-meeting-broadcast-demo https://github.com/aws-samples/amazon-chime-meeting-broadcas...

jiofih 6y ago
Is 4-5s latency considered good? It’s an awful experience for video chat.
- nowahe 6y ago
  Might have mixed up Chime & IVS a bit. To clarify, the 4-5s delay I'm talking about is for IVS, which is a live streaming solution, and doesn't have anything to do with video chat. It has a RTMP input on which you send a video stream, re-encodes it and then distribute it as an HLS stream. And getting sub 5s latency on an HLS stream is pretty fast. I brought up Chime and video chat mostly as an example on how AWS is pushing for RTMP to connect those services.
- colde 6y ago
  For live streaming distribution to loads of viewers, outside of a video meeting scenario, then yeah. That is considered good. Plenty of commercial services with latency 5-10 times that.

sneak 6y ago

> Probably the hardest part of running these things is managing outbound bandwidth costs. This might be a good time to mention that Hetzner (where I host) has a lot of unmetered gigabit options. I'm not affiliated, just a happy (long term) customer. Sadly Germany has some laws that censor certain types of harmless content, so that's one downside, but that's not Hetzner's fault, just something to consider depending on what content you intend to host.

dewey 6y ago
> Sadly Germany has some laws that censor certain types of harmless content What is this harmless content that's censored?
- sneak 6y ago
  Video game gore/violence is one example, though there are many others; German censorship law is relatively broadly worded. Games sold in Germany have to have their gore reduced (relative to versions legal in other countries) to be legal for publication in Germany. I'm not a lawyer, but my amateur assumption is that publication of video streams is subject to the same content rules as publication of game software.
nielsole 6y ago
You could use their Finnish dc. (no idea if Finnish laws are any better in this regard)
chaz6 6y ago
I can attest to Hetzner. I pay €35 per month for a dedicated server, and one of the things I run is a (non-exit) Tor relay, which consumes 3.5TiB per day on average.

supermatt 6y ago

Any specifics on DoS? What sort of strategies to prevent a malicious user pushing multiple streams to DoS encoders, etc?

ChrisMarshallNY 6y ago

I don’t like RT[M|S]P, but I have not seen an acceptable alternative for realtime streaming. The main reason I don’t like RTMP, is because it isn’t supported at the OS level in Apple systems (the platform I program for). HLS doesn’t come close to RTMP. It’s designed for movie streaming, and doesn’t support realtime streaming. I write video surveillance software, and the HLS lag is unacceptable.

zweizeichen 6y ago

Some great points! I built a project similar to this (think online lectures but it never went anywhere) in spring when the pandemic hit. I highly recommend building a platform like this if you have some spare time. It involves lots of interesting engineering/architecture challenges all over the stack. IMHO, from a technical viewpoint, it's mostly integration work, as all the really hard parts were already done by excellent 3rd party tools and libraries. You'll probably have some kind of RTMP bridge (nginx [1]), wrap ffmpeg for transcoding and play it back using something like Shaka player [2] where the segments are being served by some kind of caching reverse proxy. It took me a few weeks to get a prototype running inside a K8S cluster where oAuth-ed users could publish RTMPS streams which could be watched by other authed users via HLS in a somewhat scalable way. It was surprisingly easy to build this using Elixir/Phoenix LiveView. Some thoughts on your comment: > - Probably the hardest part of running these things is managing outbound bandwidth costs. This. As others noted, you may be getting around this by using something like Hetzner's/Scaleway's/OVH's offerings. However, I think they won't be too happy if you really use that much bandwidth all the time. You can probably forget using IaaS of one of the bigger cloud platforms unless you negotiated some special terms. > - You'll almost certainly need admin and moderation tools soon. That's one of the main reasons why I did offer a public instance. It's probably best to host a platform like this in a non-public context such as members of a single org. Just look at what happened to Firefox Send... > - Please don't keep using RTMP [...] and its security is, uh, _weak_. Yes it is a bit of a pain to work with but AFAIK you can wrap RTMP inside a TLS session which is supported by OBS. I think I just exposed a stunnel instance back then which forwarded it to a nginx instance which handled auth/auth and forwarding to the transcoding backend. This way you won't leak any streaming keys or video data. Please correct me if I'm wrong. If you have any additional pointers regarding RTMP security, I would be highly interested! Also, as others pointed out, SRT may be just around the corner. I think we're in for some interesting times as LL-DASH/CMAF and similar technologies are more widely supported. Additionally, there are interesting projects like [3] going the WebRTC route for delivering low latency streams. [1] https://github.com/arut/nginx-rtmp-module https://github.com/arut/nginx-rtmp-module [2] https://github.com/google/shaka-player https://github.com/google/shaka-player [3] https://www.ovenmediaengine.com https://www.ovenmediaengine.com

andrenotgiant 6y ago

FWIW, the OP repo uses a Terraform config to deploy the server on a 4GBRAM server on DigitalOcean.[1] Which costs $20/mon and has 4TB/mon bandwidth allowance.[2] To give a sense of what a 4TB bandwidth could cover: 320 people watching a 3hr stream at 1080p or 70mb per minute. [1] https://github.com/benwilber/boltstream/blob/master/terraform/terraform.tfvars https://github.com/benwilber/boltstream/blob/master/terrafor... [2] https://www.digitalocean.com/community/tools/bandwidth?active=%5B%7B%22slug%22%3A%22s-2vcpu-4gb%22%2C%22type%22%3A%22droplet%22%2C%22hours%22%3A744%2C%22consumption%22%3A12600%2C%22nodes%22%3A1%7D%5D&additional=0 https://www.digitalocean.com/community/tools/bandwidth?activ...

stormbrew 6y ago

"Please don't keep using RTMP." So, this has been talked about a bunch already, but I'm involved in a small streaming group that occasionally puts on (online, not in person) marathons and we've been looking for something to use that's better than rtmp for a while for ingest from participants and we really haven't found anything good. At least nothing that's viable to set up with open source software of your own. We just piloted SRT (having about 35 different streamers stream to us with it from OBS into srt-live-server, then into OBS with a media source), which has been mentioned a bunch here, and it's been a mixed bag and I don't know if we'll stick with it. The big pro is that it doesn't fall behind like the usual RTMP chain (OBS->nginx-rtmp-flv->OBS media source or web source with flvjs) tends to when connection quality degrades. I don't think this is necessarily an inherent characteristic of rtmp really, but the open software in this chain tends to just grow bigger and bigger buffers when the connection hitches, leading to progressively worse latency you have to 'fix' by resetting the stream. Twitch and other big ingesters seem to avoid this somehow, and I suspect ingesting directly into HLS or similar avoids it, but low latency HLS is still essentially unavailable in open source packaging so is not viable for our use. On the downside, though, SRT requires somewhat manual tweaking of latency and buffering parameters to work properly and our experience has been that, while the stream never falls behind, hitches outside the configured parameters produce a particularly ghastly kind of artifacting we've called "streakies" where entire segments of the feed will just turn weird bands of colours. It's possible this is actually a problem in the software, as the bits needed to put this together (particular srt-live-server) seem pretty immature yet. It seems like the future for this kind of use case is likely to wind up being webrtc, but as of now getting webrtc to play nice with OBS is kind of painful, usually just involving using external software that captures a popped out preview display from OBS. It's a little less than ideal. I think it'll be our next thing to try though, and obs.ninja looks kind of promising as a platform to help with doing it.