6 ms·

GStreamer 1.20: Embedded and WebRTC lead the way

176 points by mfilion 5y ago

CameronNemo 5y ago
I might switch to Epiphany on my Pinebook Pro due to gstreamer supporting hwaccel video decode. VAAPI (used by Firefox) does not on this hardware. I'll have to run some tests though.
- freebuju 5y ago
  You can give Chromium a try. It has unofficially supported hw video decode capability. Though in my experience, out of the available browsers, Epiphany had the most robust hw video playback support due to gstreamer integration.
  - CameronNemo 5y ago
    I am pretty sure Chromium does not have hwaccel decode on my hardware (rk3399). It uses VAAPI like Firefox.
    - freebuju 5y ago
      My bad. I thought you were saying you had issues with getting vaapi to work on Firefox. Tough luck getting any hw video acceleration in Linux outside of x86_64 and vaapi capable chips.
CyberRabbi 5y ago
These are incredible contributions to the community. Thank you Collabora and the gstreamer team.
Sean-Der 5y ago
WebRTC has so many great implementations now! The author of the C# implementation started a really great project webrtc-echoes[0] that shows them all working together. The next big challenge in the space seems to be getting widely available congestion control. The hard part is making sure it is understandable and customizable for everyones use cases. [0] https://github.com/sipsorcery/webrtc-echoes https://github.com/sipsorcery/webrtc-echoes
binarynate 5y ago
This is good news, although it looks like GStreamer's WebRTC plugin is still in its "bad" plugins repo: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/tree/master/gst-libs/gst/webrtc https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/t... From the README: > gst-plugins-bad: a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors. This was a bit of a turn off when I was evaluating GStreamer for a WebRTC based product I'm working on.
- sho_hn 5y ago
  The gst-plugins-bad repo is defunct. GStreamer now uses a monorepo (also mentioned in the article). https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/mono-repository.html?gi-language=c https://gstreamer.freedesktop.org/documentation/frequently-a... That said, webrtc is still in -bad: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/main/subprojects/gst-plugins-bad/gst-libs/gst/webrtc https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/ma...
  - binarynate 5y ago
    Thanks for the link to the new repo, I had missed that.
- Sean-Der 5y ago
  What did you end up using for WebRTC? Any specific things that you needed when looking at WebRTC implementations?
  - binarynate 5y ago
    Hi, Sean! Thanks again for creating Pion. That's what I'm using for the product now. I shared my reasons for choosing Pion on Reddit when you posted about the Pion v3 release: https://www.reddit.com/r/golang/comments/kj1net/pion_webrtc_v300_released/ggvm66l https://www.reddit.com/r/golang/comments/kj1net/pion_webrtc_...
nakovet 5y ago
I would love to do buy gstreamer course to learn more, I am finding the documentation hard to navigate on the macOS platform, it seems very linux friendly. Keeping an eye for study materials.
- deleted 5y ago
  [deleted]
- pramodbiligiri 5y ago
  Are you using their Application Development Manual? - https://gstreamer.freedesktop.org/documentation/application-development/index.html?gi-language=c https://gstreamer.freedesktop.org/documentation/application-... There’s also a Plug-in Development Guide. Both of these are good resources. PDFs of the same should be available online.
eternityforest 5y ago
GStreamer is so amazing! Probably the best way to do anything related to media.
krastanov 5y ago
I am probably asking about an apples to oranges comparison, but could someone knowledgeable help me understand in what situations I would want to use gstreamer vs ffmpeg vs pipewire vs others. They all have some overlap and can use each other for some sub-tasks, but I do not know when to reach for what.
- goombacloud 5y ago
  GStreamer is the top-level framework that allows you to connect the modules it offers to solve your problem, ffmpeg is a tool or library to invoke which may cover your case but is a bit harder to integrate and fit for arbitrary usecases, PipeWire is for live connecting input sources to output sinks at a lower level.
  - kaladin-jasnah 5y ago
    > ffmpeg is a tool or library If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.
    - Sesse__ 5y ago
      https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples ? In any case, they generally fill different niches. FFmpeg is, generally, a library containing set of A/V codecs and (de)muxers. (There are some more obscure parts, too, plus a command line tool ffmpeg(1) that gives access to most aspects of that library.) GStreamer is, generally, an A/V graph framework originally modelled after Microsoft DirectShow. If you find FFmpeg too difficult to use, most likely what you want isn't a codec library in the first place, but some higher-level abstraction (e.g. if you just want a video on screen, most likely you don't want to muck around with low-level demuxing, but you'd rather want something that deals with clocks and sync and stuff, and FFmpeg just doesn't do that—it's not a video player). It has a lot of weaknesses, but the API is fairly straightforward for what it does.
- johnny22 5y ago
preya2k 5y ago
I wanna believe that this is a fantastic piece of software. However it's really bad at describing what it does. As a software developer, I cannot find a high-level 2-5 sentence description of what GStreamer actually is/does on their website. I might just be blind, but this should really be on the home page above the fold. So many OSS projects fail at this :(
- TingPing 5y ago
  Its a multimedia framework. It can do everything in that domain, decoding, muxing, rendering, etc.
- CameronNemo 5y ago
  Their homepage does just get straight to the release notes, but if you select documentation on the sidebar you get a nice synopsis. https://gstreamer.freedesktop.org/documentation/ https://gstreamer.freedesktop.org/documentation/
- deleted 5y ago
  [deleted]
- Uehreka 5y ago
  I just deleted my reply to your comment, because I checked their homepage a second time and then saw this at the bottom: > What is GStreamer? > GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. > Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface. Read more ... > GStreamer is released under the LGPL. The 1.x series is API and ABI stable and supersedes the previous stable 0.10 series. Both can be installed in parallel. That should be at the top of the page.
- ShoveItHN 5y ago
  I agree. Apparently it's just a library, so you have to use it to construct an application. But they should provide some links to applications built with it.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]

176 points by mfilion 5y ago

I might switch to Epiphany on my Pinebook Pro due to gstreamer supporting hwaccel video decode. VAAPI (used by Firefox) does not on this hardware. I'll have to run some tests though.

freebuju 5y ago
You can give Chromium a try. It has unofficially supported hw video decode capability. Though in my experience, out of the available browsers, Epiphany had the most robust hw video playback support due to gstreamer integration.
- CameronNemo 5y ago
  I am pretty sure Chromium does not have hwaccel decode on my hardware (rk3399). It uses VAAPI like Firefox.
  - freebuju 5y ago
    My bad. I thought you were saying you had issues with getting vaapi to work on Firefox. Tough luck getting any hw video acceleration in Linux outside of x86_64 and vaapi capable chips.

CyberRabbi 5y ago

These are incredible contributions to the community. Thank you Collabora and the gstreamer team.

Sean-Der 5y ago

WebRTC has so many great implementations now! The author of the C# implementation started a really great project webrtc-echoes[0] that shows them all working together. The next big challenge in the space seems to be getting widely available congestion control. The hard part is making sure it is understandable and customizable for everyones use cases. [0] https://github.com/sipsorcery/webrtc-echoes https://github.com/sipsorcery/webrtc-echoes

binarynate 5y ago

This is good news, although it looks like GStreamer's WebRTC plugin is still in its "bad" plugins repo: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/tree/master/gst-libs/gst/webrtc https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/t... From the README: > gst-plugins-bad: a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors. This was a bit of a turn off when I was evaluating GStreamer for a WebRTC based product I'm working on.

sho_hn 5y ago
The gst-plugins-bad repo is defunct. GStreamer now uses a monorepo (also mentioned in the article). https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/mono-repository.html?gi-language=c https://gstreamer.freedesktop.org/documentation/frequently-a... That said, webrtc is still in -bad: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/main/subprojects/gst-plugins-bad/gst-libs/gst/webrtc https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/ma...
- binarynate 5y ago
  Thanks for the link to the new repo, I had missed that.
Sean-Der 5y ago
What did you end up using for WebRTC? Any specific things that you needed when looking at WebRTC implementations?
- binarynate 5y ago
  Hi, Sean! Thanks again for creating Pion. That's what I'm using for the product now. I shared my reasons for choosing Pion on Reddit when you posted about the Pion v3 release: https://www.reddit.com/r/golang/comments/kj1net/pion_webrtc_v300_released/ggvm66l https://www.reddit.com/r/golang/comments/kj1net/pion_webrtc_...

nakovet 5y ago

I would love to do buy gstreamer course to learn more, I am finding the documentation hard to navigate on the macOS platform, it seems very linux friendly. Keeping an eye for study materials.

deleted 5y ago
[deleted]
pramodbiligiri 5y ago
Are you using their Application Development Manual? - https://gstreamer.freedesktop.org/documentation/application-development/index.html?gi-language=c https://gstreamer.freedesktop.org/documentation/application-... There’s also a Plug-in Development Guide. Both of these are good resources. PDFs of the same should be available online.

eternityforest 5y ago

GStreamer is so amazing! Probably the best way to do anything related to media.

krastanov 5y ago

I am probably asking about an apples to oranges comparison, but could someone knowledgeable help me understand in what situations I would want to use gstreamer vs ffmpeg vs pipewire vs others. They all have some overlap and can use each other for some sub-tasks, but I do not know when to reach for what.

goombacloud 5y ago
GStreamer is the top-level framework that allows you to connect the modules it offers to solve your problem, ffmpeg is a tool or library to invoke which may cover your case but is a bit harder to integrate and fit for arbitrary usecases, PipeWire is for live connecting input sources to output sinks at a lower level.
- kaladin-jasnah 5y ago
  > ffmpeg is a tool or library If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.
  - Sesse__ 5y ago
    https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples ? In any case, they generally fill different niches. FFmpeg is, generally, a library containing set of A/V codecs and (de)muxers. (There are some more obscure parts, too, plus a command line tool ffmpeg(1) that gives access to most aspects of that library.) GStreamer is, generally, an A/V graph framework originally modelled after Microsoft DirectShow. If you find FFmpeg too difficult to use, most likely what you want isn't a codec library in the first place, but some higher-level abstraction (e.g. if you just want a video on screen, most likely you don't want to muck around with low-level demuxing, but you'd rather want something that deals with clocks and sync and stuff, and FFmpeg just doesn't do that—it's not a video player). It has a lot of weaknesses, but the API is fairly straightforward for what it does.
johnny22 5y ago

preya2k 5y ago

I wanna believe that this is a fantastic piece of software. However it's really bad at describing what it does. As a software developer, I cannot find a high-level 2-5 sentence description of what GStreamer actually is/does on their website. I might just be blind, but this should really be on the home page above the fold. So many OSS projects fail at this :(

TingPing 5y ago
Its a multimedia framework. It can do everything in that domain, decoding, muxing, rendering, etc.
CameronNemo 5y ago
Their homepage does just get straight to the release notes, but if you select documentation on the sidebar you get a nice synopsis. https://gstreamer.freedesktop.org/documentation/ https://gstreamer.freedesktop.org/documentation/
deleted 5y ago
[deleted]
Uehreka 5y ago
I just deleted my reply to your comment, because I checked their homepage a second time and then saw this at the bottom: > What is GStreamer? > GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. > Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface. Read more ... > GStreamer is released under the LGPL. The 1.x series is API and ABI stable and supersedes the previous stable 0.10 series. Both can be installed in parallel. That should be at the top of the page.
ShoveItHN 5y ago
I agree. Apparently it's just a library, so you have to use it to construct an application. But they should provide some links to applications built with it.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]