10 ms·
Using a Raspberry Pi to add a second HDMI port to a laptop
- pcouy 4y agoHey there ! Thank you for sharing my blog post. I tried to share it myself earlier, but my post was hidden (probably because my account is new). This is my first technical blog post. I intended for it to be easy to follow along, and tried my best to write in a pleasant style. However, as a non-native speaker, I know that my writing sometimes feels awkward. Please tell me everything that is wrong with this post, whether it's from a technical or a linguistic standpoint. Don't worry about sounding harsh, this will only help me improve :) I wish you a pleasant read.
- neilsimp1 4y agoI wish HN had more content like this. Thanks for the write-up!
- pcouy 4y agoThanks for the kind words ! Really motivates me to write more
- ta988 4y agoWhy did you use that approach and not a USB video card?
- sintax 4y agoDid you consider using Sunshine/Moonlight? https://github.com/LizardByte/Sunshine https://github.com/LizardByte/Sunshine and https://github.com/moonlight-stream https://github.com/moonlight-stream
- pcouy 4y agoI'm not at my computer rign now, and can't remember if I did, but I'll make sure to keep you updated
- xyro 4y agoMoonlight requires nvidia GPU and GeForce Experience no?
- sintax 4y agoNo, both the client (moonlight) and the server (sunshine) support AMD/Intel/nVidia. Might be useful: "System on a Chip - Benchmark for Moonlight Game Streaming Project" - https://docs.google.com/spreadsheets/d/1WSyOIq9Mn7uTd94PC_LXcFlUi9ceZHhRgk-Yld9rLKc https://docs.google.com/spreadsheets/d/1WSyOIq9Mn7uTd94PC_LX... lists the Pi3 as having about 10ms latency at 1080p 60fps.
- xyro 4y agothanks.
- manmal 4y agoWanted to write a top level comment to suggest Sunshine/Moonlight too. I've been using it to stream 3.5k@60fps from my midrange gaming PC (Ryzen 5, 3060 Ti) to my Macbook Pro. I set up a virtual display with my Macbook's native resolution (minus some height because I don't want to overlap with the notch), and it's just perfect, with no perceptible input lag. Sometimes I need to restart Sunshine, but I could just move the PC to the basement and operate it completely remotely (Moonlight supports WoL). If your host is on latest Win 10/11, you can even get by without a monitor or HDMI dummy: https://www.reddit.com/r/MoonlightStreaming/comments/rzpcpc/moonlight_streaming_without_monitor_no_dummy_plug/ https://www.reddit.com/r/MoonlightStreaming/comments/rzpcpc/...
- bmsleight_ 4y agoLove the blog post. I did something similar with Kindles - into a plug-and-play HDMI monitor. But not as good as you on delay. https://hackaday.com/2021/01/08/old-kindle-shows-hdmi-video-eventually/ https://hackaday.com/2021/01/08/old-kindle-shows-hdmi-video-... I may need to look again at the parameters you have used for ffpmeg, as it appear to be almost a sorcery to get the correct parameters.
- pcouy 4y agoIt indeed took several days of trial and errors to get a result I felt comfortable using
- jon-wood 4y agoYou totally hit the mark on being easy to follow, and pleasant to read, I love blog posts like this which don't just dump a solution and then leave it at that but also walk through the process of getting to that point. Can I suggest adding an RSS feed to your blog? I'd love to follow it, but realistically unless I can add it to a feed reader I'm going to forget it exists!
- pcouy 4y agoThanks for your encouragements. I'm still tweaking the jekyll website, but I'll make sure to add an RSS feed to it
- pcouy 4y agoHey ! I hope you didn't forget my blog already, because I added the RSS feed as you suggested. You can find it at https://pierre-couy.dev/feed.xml https://pierre-couy.dev/feed.xml
- bmicraft 4y agoYou state multiple times that the Raspberry Pi 3 does not support hardware accelerated h264 decoding, but that's simply not true. All Pis (except Pico series) support both accelerated encoding and decoding, and have done so since 2012. The Broadcom VideoCore IV in RPi 3 Model B does hw decode at 1080p30.
- pcouy 4y agoYeah you're right. Someone mentioned it on reddit and the issue was indeed not the lack of hardware decoding, but the fact that using it disabled my optimizations regarding latency. You can read more in this thread :https://reddit.com/r/programming/comments/11r9osx/using_a_raspberry_pi_to_add_a_second_hdmi_port_to/jc9bljm https://reddit.com/r/programming/comments/11r9osx/using_a_ra...
- stity 4y agoCongrats on being on the front page of HN ! Are you the Pierre Couy I met in Centrale Lyon ? We did a project together using D3.js for a pharmaceutical company. Valentin
- pcouy 4y agoYes I am, and I clearly remember the D3.js project. It has been almost 10 years, crazy how time flies! It's nice meeting you again on HN :)
- danShumway 4y agoFantastic post, great level of detail. Thank you for writing it and sharing it; I hope you write more technical stuff like this in the future.
- jacknews 4y agoAre usb video 'cards' still a thing?
- paco3346 4y agoYa, DisplayLink
- themadsens 4y agoIncidentally, this is how you add a second external monitor to an M1/M2 macbook
- 0xcde4c3db 4y agoAlthough it looks like they haven't really refreshed that product line since 2016. Seems like they're trying to pivot to wireless now that Alt Mode USB ports are mainstream.
- pcouy 4y agoFor me, the main issue was the proprietary software I needed to run un the host computer. I also read somewhere it doesn't play well with Linux. To be honest, I didn't even consider it before rolling my own solution because it was not immediately clear to me that it was something different from display port over USB
- baerrie 4y agoI was trying to find a converter for FireWire to usb-c, I wonder if an approach like this could help?
- gorkish 4y agoApple Firewire to Thunderbolt adapter plugged into Apple Thunderbolt 2 to Thunderbolt 3 adapter cable is more or less the only way to do it. You have to have Thunderbolt PCIe on the USB-C port to support the hardware requirements of firewire. Such a solution has nothing at all to do with virtual monitors or display protocols though. Firewire has a standard for transporting digital video, but it's not useful for much of anything except speaking with niche and legacy AV equipment.
- deleted 4y ago[deleted]
- talkingtab 4y agoVery helpful. I have the same situation and will give this a try. Cool!
- eurekin 4y agoThat entry was a lot more comprehensive I anticipated. Didn't find any video showing the final experience. Is there any by chance?
- pcouy 4y agoUnfortunately there is not a video yet. I'll try to add one as soon as possible
- bdcravens 4y ago> had not noticed that the laptop I chose was missing an essential feature : it did not have Display Port over USB C. Not being able to use my second external monitor on this new laptop felt like a huge downgrade from my previous one Any DisplayLink hub will solve this problem. (That's what I did with the M1 Macbook Air) Probably not as fun as hacking on a Raspberry Pi, but a lot easier to find than a Raspberry Pi :-)
- pcouy 4y agoAs I already mentioned in another comment, DisplayLink requires proprietary software on the host computer. I like my open source solution better ;) And the part of your comment about having fun hacking on a RPi is point on!
- eythian 4y agoNote that DisplayLink is awful and should not be encouraged. It works on Linux with proprietary drivers, but badly and not natively (at least on Ubuntu) and will eat your CPU. It has issues on Macs that DRMed media in the browser won't display (cos the OS sees it as a screen recorder), and it will probably eat your CPU.
- anyfactor 4y agoI like this post. As someone who is deeply afraid of their Raspberry Pi, this could be the silver bullet. An RPI project to me is 20% surface level solution and 80% troubleshooting. Your standard RPI forum thread to do one thing quickly spans to a mile long, because "I tried what you recommended but what to do now that I have a new problem". You can really feel that experiencd in this thread. Most RPI experience in my opinion is similiar to mine. You can not solve "a" problem with "a" solution. You always need a series of solutions with progressive tweaking. This post captures that essence, solving a problem and coming to a goal with an RPI that is written like a travel story.
- green-salt 4y agoI stopped using raspberry pis for this reason. A small form factor refurb PC suits what I am doing way better and I don't have to try to find ARM binaries for everything.
- pcouy 4y agoThanks for the feedback. I did not expect people to enjoy my writing that much :)
- jeron 4y agoI think it depends on the use case. My experience with my RPi used as a 3D printing tool has been plug and play with OctoPi and haven’t needed to tweak it much at all
- deleted 4y ago[deleted]
- giobox 4y ago> As someone who is deeply afraid of their Raspberry Pi, this could be the silver bullet. To me this is much the same as writing "As someone who is deeply afraid of their computer, this could be the silver bullet." A Pi is just a credit card sized ARM computer - there is no magic, it can run many of the same OSes as any other desktop or laptop, it can run the exact same apps and software in many cases. Thinking of it as somehow different is already a bad way to begin.
- yootyootr 4y agoOP, since you're in the thread: Did you install a GUI on the Pi? If not, how does the mpv or ffmpeg command know how to output video to the HDMI port?
- pcouy 4y agoNo GUI on the Pi (went with the minimal image). I don't clearly remember, but I think ffplay detects there is no X server and automatically switches to using Direct Rendering Manager. For mpv, I added a specific option to use DRM output
- ar9av 4y agoWhy would you need to do this? USB to HDMI adapters exist and now that USB-C is mainstream there are adapters for that too and significantly cheaper than a raspberry pi. One use I would like to see is a pi convert an HDMI signal to ethernet and convert it on the other side with a second pi. Lol basically any DisplayLink adapter would work. Way cheaper.
- pcouy 4y agoLol this is copy pasted from a comment on reddit...
- coreyisthename 4y agoFor what it’s worth, I can’t get any adapters to work on my ThinkPad and haven’t been able to resolve it for a long time. My only HDMI port is broken. I’m going to try this with one of the Pi’s I have laying around. I’m excited to see if I can get it to work!
- pcouy 4y agoI'm excited as well to see if you can get it to work!
- RunSet 4y agoSince you mention Thinkpad, here is a personal pet peeve. A company called StarTech makes ExpressCard devices and USB devices. They make: * An ExpressCard HDMI capture device (no longer manufactured) that sits flush with the body of the Thinkpad. [0] * ExpressCard USB3 peripherals that adds one or two USB ports to the Thinkpad and sits flush with the body of the Thinkpad. [1] * USB3 to HDMI output devices that leave an unsightly dongle protruding from the Thinkpad. [2] So, what StarTech does not make (and my pet peeve): An ExpressCard device that adds an HDMI output port to a laptop and sits flush with the laptop's body. Note that such a product would seem to be a trivial combination of their existing products. I would also expect that such a device would be cheaper to manufacture than the equivalent capture device. [0] https://www.startech.com/en-us/audio-video-products/echdcap https://www.startech.com/en-us/audio-video-products/echdcap [1] https://www.startech.com/en-us/cards-adapters/ecusb3s254f https://www.startech.com/en-us/cards-adapters/ecusb3s254f [2] https://www.startech.com/en-us/display-video-adapters/usb32hdes https://www.startech.com/en-us/display-video-adapters/usb32h...
- spants 4y agousb-c to hdmi adapters are super cheap now.
- pcouy 4y agoI know, I've already answered this point in other comments
- realworldperson 4y ago[dead]
- emersion 4y agoThere is an upstream KMS driver for GUD: https://github.com/notro/gud https://github.com/notro/gud It works by connecting the Pi via USB.
- MayeulC 4y agoYeah, I immediatly thought of this. They even have pre-built images. I wonder what OP uses as a DE. In sway (and presumably other wlroots-based compositors) it's very easy (one command) to add a headless output. Edit, as I saw your username: sadly it's undocumented in the man page, IIRC :P
- pcouy 4y agoSomehow I did not come across this when researching existing solutions. Looks interesting though, I'll look into it
- mdip 4y agoExcellent write-up. I love these kinds of "I had this problem, these things on hand and an idea for how to solve it" followed by the story of every barrier and how it was overcome. It's becoming more and more rare with people preferring to go the YouTube route. I'm not knocking video-versions of these things, they have a lot of value (among them being "less friction to publish easily" for some folks[0]) but convenience of text is vastly superior for consumption as far as I'm concerned. It takes a lot of time and energy to put these sorts of things together but they have value beyond the original topic (e.g. perhaps one of those barriers you encountered designing this is a barrier to other problems[1]). [0] Sometimes that's perception, sometimes it's "I don't type very quickly/use a phone exclusively for my computing needs" [1] Unexpectedly, your site produced a TLS error in Firefox on one of my laptops, so it's causing me to investigate a problem I didn't know I had :)
- relaxing 4y agoI'd much rather type out the instructions than stage and edit video of myself doing it. And I'd much, much rather read at my own pace than be forced to watch at someone else's pace.
- kevin_thibedeau 4y agoInstructional videos are best watched at 2x speed.
- narwally 4y agountil you get to the one actually tricky part and you need to pause it for sec so your brain can catch back up.
- nerpderp82 4y agoUse AI to dynamically time warp the video relative to the information density in the video. Fast through the boilerplate, easy to understand parts, slow through the hard parts.
- strobe 4y agoI thought about that in different angle - does it possible to use a second linux laptop as external monitor for a mac. I've tried few similar streaming solutions without investing much in tuning and creating proper wire connection and as result my initial impression was that is too much latency issues (as well with some additional issues related to creating a virtual screen), but probably with your approach I'll try again. Thanks.
- theturtletalks 4y agoI'm using a capture card to use my Macbook monitor as a screen for my PS4 when I travel. You could use one to turn your Linux laptop into an external monitor.
- strobe 4y agohm, that really nice solutions - what capture card do you have?
- mbreese 4y agoI’ve done this the opposite way round, using a cheap USB C capture dongle from Amazon to capture output from an RPi and then using my a Mac as a virtual screen for the RPi. I do this occasionally when I need to setup something but don’t want to drag out an external monitor. The capture card/dongle are designed for using in Twitch game streaming where you are using it to capture the output from a game and feed it into something like OBS. The one I have was pretty cheap, but only supports an output resolution of 1080p (input up to 4K). I don’t remember the exact brand, but the above should give you enough keywords to get started.
- theturtletalks 4y agoElgato HD60X: https://www.elgato.com/en/game-capture-hd60-x https://www.elgato.com/en/game-capture-hd60-x There are cheaper options if input lag isn’t an issue.
- johnwalkr 4y agoBarrier will let you use your mac’s keyboard and mouse on the second linux laptop. Depending on your use case that might be good enough. I actually prefer this as I can avoid opening YouTube etc on my work laptop.
- xdanger 4y agoI did something similar for my macbook with uxplay: https://github.com/FDH2/UxPlay https://github.com/FDH2/UxPlay I ran it in console directly rendering to framebuffer and it has hardware-accelerated video decoding. On Raspberry pi 3, hooked to lan and my macbook on wifi the lag is ok. It's about the same as ssh over slow cellular. I mainly run my terminals watching the logs/builds etc. on that display.
- pcouy 4y agoThe hard part for me was getting the latency low enough that I can comfortably type and move the mouse. But I may look into is as a 4th screen for logs would be interesting
- xdanger 4y agoJust to let people know how to get the latency down with uxplay: Upgrade the debian to bookworm (uxplay is in the repos for that one) apt install uxplay in in /boot/config.txt change doverlay to: dtoverlay=vc4-kms-v3d force hdmi: (i had some problems with kms) hdmi_force_hotplug=1 add gpu mem: gpu_mem=256 then run: (-a disables audio) uxplay -vs kmssink -a and now you have and airplay monitor that can mirror or extend your desktop.
- steve918 4y agoGiven the cost and availability of RPi at the moment this is an expensive option.
- pcouy 4y agoAhah true! But I had one lying around so for me it was essentially free
- dmitrygr 4y agoOn one hand, this is a GREAT example of screws looking like nails when all you have is a hammer. On the other hand, this was a VERY impressive way to screw in the screws using a hammer! A tip of my hat to you, sir.
- pcouy 4y agoThank you for actually making me laugh out loud ! :) Makind this indeed felt like hammering on a screw at times.
- rcarmo 4y agoThis is nice, but has too many moving parts and ffmpeg-like approaches are _not_ the way to go for low-end desktop streaming--I tried doing that in unicast and multicast scenarios quite a few times, and have a few notes on the results: https://taoofmac.com/space/hw/raspberry_pi/streaming https://taoofmac.com/space/hw/raspberry_pi/streaming Instead, I would have used Xrdp (which also works with xrandr) and the DirectFB RDP client instead, like I did ten years ago: https://taoofmac.com/space/blog/2013/01/09/2339 https://taoofmac.com/space/blog/2013/01/09/2339 Even that far back, it worked _extremely_ well on a single-core Pi, and I suspect it would fly on a Pi 3. I used one recently to build a GPU-accelerated thin client: https://taoofmac.com/space/blog/2022/08/14/2030 https://taoofmac.com/space/blog/2022/08/14/2030 ...which I recently took to the next level by using xorgxrdp-glamor on the server side: https://taoofmac.com/space/blog/2022/10/23/1700 https://taoofmac.com/space/blog/2022/10/23/1700 I think DirectFB already supports RemoteFX/GFX AVC:444 and all the other niceties that you get with xorgxrdp-glamor on the server side, and am willing to bet the whole thing would run a lot _faster_ and more efficiently than this solution all around, power and bandwidth-wise. (this is being written in one such RDP client against a Fedora server, and I have YouTube open with audio streaming to the client... Which also works OK via Wi-Fi)
- pcouy 4y agoLooks really interesting. I'll make sure to check your notes and plog posts out.
- shrx 4y agoA long time ago when I was looking for a low latency solution for streaming _from_ the Pi (should also have a similar performance in the other direction), gstreamer[1] was the only usable option. [1] https://gstreamer.freedesktop.org/ https://gstreamer.freedesktop.org/
- rcarmo 4y agoThe Pi is _much_ better at decoding than encoding, though.
- raffraffraff 4y agoHow about using socket activated service on the pi to receive data over a port? That's could automatically handle turning on/off the display whenever the service starts. It's a while ago and I don't use it any more but I did this, to send audio to my pi: https://github.com/raffraffraff/audio-teleport https://github.com/raffraffraff/audio-teleport
- pcouy 4y agoThat would be one nice way make the whole thing cleaner on top of turning it into a deb package. For now, it's really hacky but has proved to fulfill my needs. I may or may not take the time to polish this up in the future. In the meantime, I've licensed the repo under GPL so if someone feels like doing it, they're more than welcome to.
- ChancyChance 4y ago@signa11 If you are running *nix on your laptop, why not just run an Xserver on the Pi with the dual screen setup and use DISPLAY=server:0 to render to it? I'm a bit confused why have the Xserver running on the laptop.
- pcouy 4y agoI don't think that would allow me to seamlessly move workspaces and windows between monitors. From the top of my head, your suggestion would imply the Pi running it's own desktop environments and having its own input devices
- ChancyChance 4y agoI thought the point of an Xserver is that it has no input devices, it simply receives instructions about cursors and rectangles and text, no? Perhaps I do not understand X.
- pcouy 4y agoWhat i meant when mentioning input devices is that the Pi would have it's own cursor, and I would have needed to handle the forwarding of inputs from the laptop's X server to the Pi's. This would have been a lot more difficult as I would have had to forward the inputs to the Pi only when moving the cursor out to the Pi-controlled monitor. Keep in mind that I'm writing this from the top of my head, without googling anything or experimenting with the setup we're talking about. It's totally possible that I'm wrong. Anyway, the hard part in this project was optimizing everything. The first proof of concept stream was pretty straightforward to get. Getting a slow PoC of what you described should be kind of easy as well. I encourage you to experiment yourself with the setup you suggested, you'll learn a lot more than you could possibly learn by reading a writeup. If you ever try it out, I'd love to hear about how it went.
- ChancyChance 4y agoSure will. My comments didn't mention the fact that I enjoyed reading your post and following your systematic analysis.
- metadat 4y agoOne thing I would really appreciate any clarification on: How did they create the virtual desktop that the mouse and keyboard could interact with as if it were a physical screen off to the side? Edit: Wow @pcouy, thanks for the sub 2-minute reply! Cheers~ and kudos to you for creating a fantastically easy to read and entertaining writeup.
- pcouy 4y agoThis is done by configuring the X server on the host computer. The method I used only works with Intel integrate graphics using Linux though.
- dschulz 4y agoVery interesting! The choice with the subnetting is a bit odd to me, it's surprising that you got the hosts talking to each other with that addressing. I would have chosen a proper /30 between the two computers, that would be 10.0.0.1/30 and 10.0.0.2/30. 10.0.0.0 network address 10.0.0.1 host (usable) 10.0.0.2 host (usable) 10.0.0.3 broadcast address Nice writeup nonetheless!
- pcouy 4y agoYeah, someone else mentioned this on reddit. When taking the time to think about it, I as surprised as you are that it worked. I'll update the post when I have more time to take care of all the things people suggested
- rainbowzootsuit 4y agoDid you ever try a smaller GOP after you patched it with the 300 frame interval? -g 1 would give a keyframe every frame so I'm wondering if the -g 100 holdover might be counterproductive once you'd done your other tweaks. Right now you have an I frame followed by 99 P-frames which could lead to a flake out for 3s if you ever lost an I-frame. The P-frames are more efficient than I-frames, but at 50mbit you don't need to set your compression too aggressively. I think I'd try (and I know your project is done) some flags along the lines of -vcodec libx264 -preset ultrafast -tune zerolatency -g 1 https://trac.ffmpeg.org/wiki/StreamingGuide#Latency https://trac.ffmpeg.org/wiki/StreamingGuide#Latency There's some other tweaks in there for the ffplay side to reduce its latency. Thanks for the nice write up.
- tamiral 4y agothis is great work ! i am going to tinker and try it out with one of my spare raspberry pi's this weekend!!!!!
- pcouy 4y agoI'd love to get your feedback (in Github discussions or issues, depending on your results), especially regarding the installation scripts. I somewhat tested them but did not take the time to start fresh and check if they actually take care of everything
- brokenmachine 4y agoVery interesting write-up, thanks for the effort. I love reading about little projects like this.