5 ms·

It's not clear to me what Pipewire does with respect to video. The audio part makes sense, most talk of pipewire that I've seen focuses on using pipewire as a

It's not clear to me what Pipewire does with respect to video. The audio part makes sense, most talk of pipewire that I've seen focuses on using pipewire as a replacement to pulseaudio. But what exactly does it do for video? I assume pipewire doesn't replace video playing applications like mpv. Would mpv use pipewire in place of some other component? Would/could mpv use pipewire in place of ffmpeg? Or is pipewire for video something like a wayland compositor?

papercrane 5y ago
The original use case for pipewire was video, it was originally named "PulseVideo". It allows things like using an application running on Wayland as a video source and piping it into Firefox so you can share your screen on Zoom or something similar.
Accacin 5y ago
I'm not sure if you've read this or not, but there was a little information on the Arch wiki, here: https://wiki.archlinux.org/title/PipeWire#WebRTC_screen_sharing https://wiki.archlinux.org/title/PipeWire#WebRTC_screen_shar...
Arnavion 5y ago
The fact that it's a filter graph for shuttling buffers between sources and sinks lends itself to both buffers for audio and buffers for video. An application that generates video frames could decide to send its video frames to pipewire, and an application that accepts video frames could accept them from pipewire. This is how screensharing through xdg-desktop-portal (linked in my sibling comments) works. The xdp implementation gets screencapture frames from the compositor and pushes them into pipewire, and the screensharing application (browser, etc) pulls the frames from pipewire. This isn't new; DirectShow on Windows, and its successor Media Foundation, are also based on graphs. Put "windows graphedit" and "windows topoedit" in an image search engine for examples.
venamresm__ 5y ago
Arnavion got it right, but a simpler case is that you would be able to share your webcam or video streams with multiple applications at the same time, while also allowing you to insert filters or modification of that video on the fly. Similar to how you can do that with audio servers right now.
patrickthebold 5y ago
this PR https://github.com/obsproject/obs-studio/pull/4287 https://github.com/obsproject/obs-studio/pull/4287 suggests OBS uses it for wayland screen capture.
zxzax 5y ago
It does the same thing it does for audio: routes and multiplexes streams. Video devices in Linux the same problem as audio devices where multiple applications can't have them open at the same time. Pushing the video through a daemon also allows you to do the same interesting things that you can do with audio streams in pulseaudio/jack, where you can transparently change the inputs and outputs around without having to restart or reconfigure the application, and you can route video through arbitrary effect chains before it reaches the destination. If the application can use v4l devices directly (I think this includes all ffmpeg players such as mpv) or can output a video stream somewhere, then it might make sense to also add a pipewire backend there. Maybe somebody will make this easy by adding a pipewire encoder/decoder to ffmpeg? Then the applications won't have to do much at all to support this. If the application uses gstreamer, you may already be able to get it to work automatically by using pipewiresink/pipewiresrc.
mxmilkiib 5y ago
One can do modular audio synthesis with JACK+LV2; https://github.com/mxmilkiib/carla-lv2-jack-nsm https://github.com/mxmilkiib/carla-lv2-jack-nsm wtay has mentioned that PipeWire will be able to allow for system-wide video synthesis; https://youtu.be/PIgPIrYq-Pc https://youtu.be/PIgPIrYq-Pc See also https://github.com/mpv-player/mpv/issues/8569 https://github.com/mpv-player/mpv/issues/8569 And https://gitlab.freedesktop.org/ryuukyu/helvum https://gitlab.freedesktop.org/ryuukyu/helvum

papercrane 5y ago

The original use case for pipewire was video, it was originally named "PulseVideo". It allows things like using an application running on Wayland as a video source and piping it into Firefox so you can share your screen on Zoom or something similar.

Accacin 5y ago

I'm not sure if you've read this or not, but there was a little information on the Arch wiki, here: https://wiki.archlinux.org/title/PipeWire#WebRTC_screen_sharing https://wiki.archlinux.org/title/PipeWire#WebRTC_screen_shar...

Arnavion 5y ago

The fact that it's a filter graph for shuttling buffers between sources and sinks lends itself to both buffers for audio and buffers for video. An application that generates video frames could decide to send its video frames to pipewire, and an application that accepts video frames could accept them from pipewire. This is how screensharing through xdg-desktop-portal (linked in my sibling comments) works. The xdp implementation gets screencapture frames from the compositor and pushes them into pipewire, and the screensharing application (browser, etc) pulls the frames from pipewire. This isn't new; DirectShow on Windows, and its successor Media Foundation, are also based on graphs. Put "windows graphedit" and "windows topoedit" in an image search engine for examples.

venamresm__ 5y ago

Arnavion got it right, but a simpler case is that you would be able to share your webcam or video streams with multiple applications at the same time, while also allowing you to insert filters or modification of that video on the fly. Similar to how you can do that with audio servers right now.

patrickthebold 5y ago

this PR https://github.com/obsproject/obs-studio/pull/4287 https://github.com/obsproject/obs-studio/pull/4287 suggests OBS uses it for wayland screen capture.

zxzax 5y ago

It does the same thing it does for audio: routes and multiplexes streams. Video devices in Linux the same problem as audio devices where multiple applications can't have them open at the same time. Pushing the video through a daemon also allows you to do the same interesting things that you can do with audio streams in pulseaudio/jack, where you can transparently change the inputs and outputs around without having to restart or reconfigure the application, and you can route video through arbitrary effect chains before it reaches the destination. If the application can use v4l devices directly (I think this includes all ffmpeg players such as mpv) or can output a video stream somewhere, then it might make sense to also add a pipewire backend there. Maybe somebody will make this easy by adding a pipewire encoder/decoder to ffmpeg? Then the applications won't have to do much at all to support this. If the application uses gstreamer, you may already be able to get it to work automatically by using pipewiresink/pipewiresrc.

mxmilkiib 5y ago

One can do modular audio synthesis with JACK+LV2; https://github.com/mxmilkiib/carla-lv2-jack-nsm https://github.com/mxmilkiib/carla-lv2-jack-nsm wtay has mentioned that PipeWire will be able to allow for system-wide video synthesis; https://youtu.be/PIgPIrYq-Pc https://youtu.be/PIgPIrYq-Pc See also https://github.com/mpv-player/mpv/issues/8569 https://github.com/mpv-player/mpv/issues/8569 And https://gitlab.freedesktop.org/ryuukyu/helvum https://gitlab.freedesktop.org/ryuukyu/helvum