11 ms·

FarPlay: Low-Latency Audio over the Internet

204 points by sp332 5y ago

hunterb123 5y ago
Does this work between two devices on the same LAN? - edit - Thanks to another user I found: https://www.sonobus.net/ https://www.sonobus.net/ Which does seem to work over LAN.
- Shared404 5y ago
  I'm curious, what's the use case here? Not that it's wrong to do or anything, just intrigued.
  - jeroenhd 5y ago
    Not the parent post, but I've been thinking about unifying all devices in my home by sending all audio streams into a single, unified "sound server" from which I can easily switch between TV audio, Bluetooth, headphones, etc. The idea is to be able to take a call on my phone and receive notifications on my laptop while developing on my PC. It'd also be nice for streaming stuff in general. As many people who have also looked into this stuff probably already know, interactive audio is really sensitive to latency. I've given it a quick shot by running pulse over the network, but honestly I wasn't impressed. I've given up for now, but some suggested software here might be good enough for a second shot, based on pipewire and its JACK backend this time.
    - getcrunk 5y ago
      Hear hear. I was just thinking about a putting all my music and a mpd on my pi then running clients on tv/computers/phones. But ur idea is the next step!
      - Shared404 5y ago
        > I was just thinking about a putting all my music and a mpd on my pi then running clients on tv/computers/phones I actually do something similar to this to run music for an online D&D campaign I'm in, except I use the http output so none of the other players have to install a client. I do like the idea of the home audio server use case, I think it'll go on my back burner!
    - smaudet 5y ago
      That sounds too connected - I like the idea of some sort pulseaudio network services but with better latency. I'd like to see pulseaudio with a better cross platform UI and a set of decentralized services versus introducing a single point of failure like you are proposing. Maybe a solution with dns-sd with stun support and a slick UI for desktop/mobile?
    - iostream23 5y ago
      Or AVB as I note above, depending upon your lan and hardware
  - hunterb123 5y ago
    My wife and I play video games together on separate PCs. Our computers are beside each other so we need to use headsets, it's important to hear 360 audio for footsteps and other sounds in shooters. We need to hear each other, game audio, teammates voices, and each other's voice. Our voices can sometimes bleed through the headsets so we need there to be little to no lag to reduce reverb. Plus the faster our voices reach each other the faster we can react to the command / callout. We can't use voice channels in the game, discord, or anything that goes over the internet. Currently we use teamspeak with a local server, which works well, looking to reduce it more. Sometimes we simply play with one headphone ear partially off, but that ruins the surround sound.
    - Shared404 5y ago
      Ah, makes sense, thanks for satisfying my curiosity!
    - Ugohcet 5y ago
      You probably should try to do it old analog style: just plug your microphones into each other’s PCs and set up soundcard's mixer to add mic input into the mix
    - Moru 5y ago
      Did you try a local mumble server? Haven't played with it for a long time but last time it was much faster than teamspeak and we used it happily at home sitting next to eachother. That and get better headphones that close out sound from environment :-)
      - hunterb123 5y ago
        Mumble was a bit slower for us than TS3 actually. Around 40ms avg where TS3 is 15-30ms.
- iostream23 5y ago
  Well, there’s actually a standard for low latency LAN audio that uses PTP “precision timing protocol” a frame prioritization protocol. Given a switch that supports PTP, including some bog-standard Netgear models, you can use what is called AVB or audio video bridge. You can use an AVB driver directly on each system and audio being processed on one system can playback over another’s audio hardware! So, there’s also a layer three standard called AES67 from the Audio Enfineering Society and this competes directly with systems like Dante. I have no idea how these perform over the broader general internet as I would think that each hop alters the latency landscape significantly
dantepfer 5y ago
Hi everyone, Dan Tepfer here, co-creator of FarPlay. As a skeptic myself, I appreciate the skeptical tone of this thread. And I'd like to address a few points made here about FarPlay and low-latency audio. First, a little about me: I've been coding for most of my life (see https://www.youtube.com/watch?v=SaadsrHBygc https://www.youtube.com/watch?v=SaadsrHBygc for my NPR Tiny Desk Concert of my improvised algorithmic music project Natural Machines) but I'm first and foremost a musician. I make my living playing concerts around the world as a pianist. During the pandemic, I used JackTrip to perform remote livestream concerts with some of the greatest musicians in jazz: Christian McBride, Cécile McLorin Salvant, Ben Wendel, Gilad Hekselman, Fred Hersch, Antonio Sanchez, Melissa Aldana, Miguel Zenon, Linda May Han Oh and others. This is just to say that music, and particularly rhythm, is very important to me, and that I care about low-latency audio as an active practitioner. Someone in this thread wrote that for rhythmic (groove) music, latencies of 3ms are noticeable, and latencies higher than 6ms are prohibitive. This isn't the case. Sound travels in air at about 1ft per ms, so a latency of 3ms is equivalent to playing with someone 3ft away from you, which is obviously unnoticeable. 6ms is equivalent to playing with someone 6ft away, which is also unnoticeable. James Brown grooved his ass off with his band spread out over a relatively wide area on stage, long before in-ear monitors, which confirms what the research says: even for advanced professional musicians, latencies up to 20ms (equivalent to 20ft in air) are not significantly noticeable even for intricately rhythmic music. Here's an excerpt over JackTrip with Christian McBride, where at the end, we play a demanding bebop head in unison, a very tough test of latency: https://www.facebook.com/watch/?v=1076063889493342 https://www.facebook.com/watch/?v=1076063889493342. Above 20ms, things do get noticeable, but depending on the type of music you're playing, it's possible to adjust. It starts to feel like the people you're playing with are, as we say in the jazz world, "laying back on the beat". For example, I did a livestream performance for the French Institute in NYC last January with pianist Thomas Enhco in Paris and myself in Brooklyn, 3500 miles away, and despite a clearly noticeable (to us) ~40ms of latency, we were able to make real music together, including rhythmic music. Note that at the time, using JackTrip, I couldn't accurately estimate the actual latency, and this 40ms figure is a guess. FarPlay, in contrast, measures the current latency and displays it on the connection screen. Someone mentioned Jamulus. I've tried Jamulus, and for my professional needs, which include rock-solid stability and the lowest possible latency, JackTrip is far superior. But JackTrip, as someone else pointed out here, is impossible to use for the average user. It requires opening ports on your router, interacting with the command line, and installing and using Jack, which itself is forbidding for most users. Our goal with FarPlay was to take the best elements of JackTrip, unbeatable stability and latency, and make them easily accessible. SonoBus is also mentioned in this thread. SonoBus is an excellent project which we only came across a few months ago. We've tried it, and we've found that if you measure the actual sound-to-sound latency, i.e. the time from sound production at the source to sound reproduction at the destination, FarPlay achieves lower latencies than SonoBus, probably because of the way it processes audio internally. Also, we believe our interface, which we've put a lot of thought into, is easier to use for non-technical musicians than SonoBus. Another advantage of FarPlay over SonoBus, this one particularly important to me as a live performer, is Broadcast Output, which is an essential feature of JackTrip that FarPlay co-creator Anton Runov and I invented (see https://farplay.io/about#history https://farplay.io/about#history). To play in low latency, it's often necessary for the musicians to tolerate artifacts in the audio, since some audio packets inevitably get delayed on their way. Broadcast Output allows you to play in low latency with artifacts in your headphones, while simultaneously outputting artifact-free audio for live broadcast or recording. To me, this is the holy grail of remote performance, allowing us to have our cake and eat it too — ultra-low-latency interaction with no sacrifice in final audio quality (see https://farplay.io/tipsandtricks#broadcastoutput https://farplay.io/tipsandtricks#broadcastoutput). I should mention that FarPlay only allows one-to-one connections at the moment, while SonoBus allows multi-user sessions. We plan to add multi-user sessions to the FarPlay user interface soon; our underlying processes already allow them. Some of you are nitpicking our claim to not require third-party software. Remember, we're coming from JackTrip, which requires users to install and use Jack in addition to JackTrip. On Mac and Linux, there is no third-party software whatsoever required. On Windows, low-latency audio is currently impossible without ASIO drivers. Many musicians on Windows have audio interfaces with ASIO drivers already installed, so in their case there are no additional downloads required. If you don't have an ASIO driver, you'll have to use ASIO4ALL, but this true for any software doing low-latency audio on Windows. In essence, FarPlay is as self-contained as it can be at this stage. Someone asked if FarPlay will connect two users on the same LAN. The answer is yes, it works great. Someone else brought up the advantages of low-latency audio not only for music, but also for regular conversations. We wholeheartedly agree: conversations feel vastly more natural without the awkward delay added by Zoom, FaceTime, WhatsApp and regular phone calls. FarPlay also transmits uncompressed audio, so the quality is as good as your mic and sound card can provide, which also helps conversations feel more real. In conclusion, I want to thank you for bringing your attention to FarPlay, and if you enjoy playing music with other people, we'd love for you to try it! It's really quite magical, I feel, and the magic hasn't worn off for me even after having done it regularly for over a year. We've tried to make the process of using FarPlay as frictionless as possible: you don't even need to register for an account to use it, just download it (https://farplay.io/download https://farplay.io/download) and go. Thanks and Happy Thanksgiving to those of you who celebrate, --Dan Tepfer https://dantepfer.com https://dantepfer.com
brrrrrm 5y ago
The documentation say this is based on JackTrip, code here: https://github.com/jacktrip/jacktrip https://github.com/jacktrip/jacktrip
- fredguth 5y ago
  Jacktrip is really hard do set up in your non-tech musician friend's computer.
  - sc20910 5y ago
    @fredguth - thats actually the reason for the MTA Easybox - a jacktrip appliance needing no knowledge to run at all. Contact networksound.com for info
ReactiveJelly 5y ago
> You don't need to install any third-party software Oh, so this is like a web browser thing? > Click here to download FarPlay Oh, so I _do_ need to install FarPlay. Just not any software that's a third party besides FarPlay. Which wouldn't make any sense.
- tyingq 5y ago
  >Just not any software that's a third party besides FarPlay. Which wouldn't make any sense. Heh. "Note for Windows: To use FarPlay on Windows, you must have an ASIO audio driver. We recommend the free ASIO4ALL."
  - akiselev 5y ago
    Coming soon to a Windows automatic update near you, next time you're running a business critical overnight batch job.
  - deleted 5y ago
    [deleted]
  - smoldesu 5y ago
    Hope you're not planning to play with a backing track, ASIO4ALL notoriously does not play nice with multiple audio sources. It's almost like someone wanted to backport the horrors of ALSA to Windows because they missed how annoying it was having a single pair of inputs and outputs.
    - whyoh 5y ago
      >ASIO4ALL notoriously does not play nice with multiple audio sources. Yep, that's because Asio4all uses WDM-KS and WDM-KS since Vista doesn't support multiple sources. Actual ASIO drivers made by sound card manufacturers usually don't have this limitation, as long as you keep the same sample rate everywhere. But it can also vary depending on who made the driver and/or on whether the source apps are using ASIO or a mix of ASIO and WDM/WASAPI. Getting low latency audio to work nicely on Windows can be messy (compared to macOS, at least).
tonymet 5y ago
how can it support a p2p socket without opening ports ? i’m confused
- sp332 5y ago
  Stateful firewalls let UDP traffic out. And they generally let "replies" come back from the destination IP & port to the source IP and port on the host. That means that you just need a central server to coordinate the session ID and the IP & port of each side. Then the two peers can send each other data directly and the firewall will let it through because it's a "reply" to the data that was sent.
- davidhyde 5y ago
  I think they are referring to opening ports on your router as opposed to the automatic opening of incoming ports when connections are opened from the client machine. They probably use the STUN protocol to facilitate connecting two machines together peer-to-peer like what is done with WebRTC.
- spacechild1 5y ago
  See https://en.m.wikipedia.org/wiki/UDP_hole_punching https://en.m.wikipedia.org/wiki/UDP_hole_punching
gonesilent 5y ago
see also ninjam if you can't get over lag because of distance limitations.
adar 5y ago
I've been looking for a simple way to be able to listen to audio from my desktop and laptop simultaneously without requiring a hardware mixer or something. I'm currently using Scream but there is a bit of latency/delay. Farplay seems to be more for audio recording, but would Farplay be a good solution for my use case?
- smegsicle 5y ago
  shouldn't VLC make audio sync easy?
- Kubuxu 5y ago
  If you are on Windows and in a single network take a look at Voicemeeter and VBAN: https://vb-audio.com/Voicemeeter/banana.htm https://vb-audio.com/Voicemeeter/banana.htm
  - adar 5y ago
    This is a good tip but unfortunately I'm on Windows on one computer and Fedora on the other.
- dantepfer 5y ago
  This isn't a use case that we've considered, but FarPlay would work well for this. If you want to play a file on one computer and have it play at the same time on the other, you'll have to play the file into FarPlay by using something like BlackHole, Jack or LoopBack, depending on your platform.
plusCubed 5y ago
What's the difference between this and Sonobus [1], which is open source and also supports iOS/Android? [1] https://www.sonobus.net/ https://www.sonobus.net/
- FractalHQ 5y ago
  Sonobus is incredible, it also ships with VST plugins for routing audio tracks from a DAW into a room.
deleted 5y ago
[deleted]
smoldesu 5y ago
Why does a peer-to-peer app like this need a subscription?
- deleted 5y ago
  [deleted]
- discordance 5y ago
  Generating those sessions to punch through NAT will need some sort of backend service, but I don't think it's subscription worthy. I would be happy to pay for this if it were a once off once the beta is over, and if FarPlay needs some dev money for upcoming features or functionality, then I would be happy to pay for the updates as well instead of a subscription.
  - WatchDog 5y ago
    Mozilla used to run a free STUN service, but it looks like they killed it after it after it was abused to conduct certain attacks[0]. [0]: https://bugzilla.mozilla.org/show_bug.cgi?id=1324520 https://bugzilla.mozilla.org/show_bug.cgi?id=1324520
- oliverevans96 5y ago
  Not all clients have an available P2P route between them, and require a dedicated TURN server to forward the traffic between them. So there are some ongoing compute costs. But they also put the upfront effort into writing the code, and there is an ongoing maintenance burden. And they're looking for a way to turn a profit on their work.
rectang 5y ago
Oh good golly yes please we need innovation in this space. I'm so tired of having cell phone calls and video calls with long latencies. Latency is so disruptive to the normal flow of conversation. If there was a reliable low-latency alternative I would try to hold all my regular convos over it, starting with family calls and proceeding to work and professional calls if feasible.
- agret 5y ago
  Mumble is pretty great for low-latency audio. TeamSpeak3 too.
  - Moru 5y ago
    Less great for setting up at your it-challenged friends though. And most people expect some phone call interface that beeps at them when they get a call.
  - jagger27 5y ago
    When Discord first launched and my usual TeamSpeak friends moved over there I was super annoyed by the extra latency. How a group of fairly serious gamers who otherwise complained about less lag in any other circumstance shrugged it off, I'm not sure. More recently, I was surprised by low latency was between two Asterisk servers in the same city on different ISPs. It was a very adhoc setup with a cheap EOL Cisco IP phone on either end connected to an Asterisk SIP server on the local network. I'm so used to laggy voice chat that it actually caught me off guard how nice it was once I actually got it to work. Besides being a total nightmare to configure. I struggle enough in person to find the right time to talk without interrupting, and >100ms of Discord latency makes it that much worse for me. I hope something peer-to-peer like this catches on for remote teams. I could really benefit from it. I don't think I would mind if a screen share lagged behind the speaker's voice. I'm sure Zoom already does that though.
    - WolfRazu 5y ago
      You should be getting considerably less latency than that. Is the Discord server perhaps set to the wrong region?
raggi 5y ago
Preface: the product is probably fine, and certainly provides a nice simple UI which is perfect for it's intended use cases. There is something fishy about the video, though. You see in the video that Zoom has been keeping audio & video in sync. In the first latency demo (https://youtu.be/Zju8YaRcSk4?t=50 https://youtu.be/Zju8YaRcSk4?t=50), we see not only latency on the audio, but also latency on the video. In the second latency demo (https://youtu.be/Zju8YaRcSk4?t=225 https://youtu.be/Zju8YaRcSk4?t=225), once FarPlay is handling audio and they've muted Zoom, magically they've managed to synchronize the video. Was this done post-edit for some reason?
- axiosgunnar 5y ago
  Yes, it's displayed in the video (as a text overlay) as an info text
  - raggi 5y ago
    Oh so it does (https://youtu.be/Zju8YaRcSk4?t=217 https://youtu.be/Zju8YaRcSk4?t=217), I missed it :)
axiosgunnar 5y ago
Would using something like this improve the quality of typical business meeting calls? E.g. reduce people talking over each other, reduce mental stress caused by the delay? Compared to Whatsapp/Zoom/etc
- sp332 5y ago
  It currently only supports two nodes connecting. The older jacktrip code supports more nodes, and FarPlay says this feature is coming later. But yeah, this would help a lot I think.
rdtennent 5y ago
"No dedicated hardware"; but for best results you'll need a wired ethernet connection, wired headphones, a good microphone and an audio interface, and on Windows an ASIO driver. "For the lowest possible latency, FarPlay establishes peer-to-peer connections between users": Except when the peers are on different ISP networks and the packets are forced to travel to some distant node, introducing latency at every node along the way. In my experience with this issue using Jamulus, having a server in the cloud at a suitable location reduces latency. Minimizing "the distance travelled" isn't important if there are few nodes introducing latency. The speed of light along wires is orders of magnitude faster than the speed of sound through air. "the faster your connection, the better your results will be"; Misleading; it's latency not bandwidth that's critical and a "fast" connection normally refers to bandwidth. Anyone interested in audio over the internet should check out Jamulus at https://jamulus.io/ https://jamulus.io/
- tekstar 5y ago
  Agree on jamulus. I re-joined my highschool band during the pandemic, despite us all living in different cities. Jamulus jams were an important part of us getting back together, along with zoom calls and passing recordings back and forth. Best thing that happened for me out of the pandemic.
  - gavinray 5y ago
    If anyone is looking for realtime collaboration tools, there's a free (proprietary) one that has live video + audio chat and a shared project workspace for building songs. https://beatconnect.com/ https://beatconnect.com/ The homepage does a pretty poor job of showing what it is, so it'll probably make more sense if you just see it: https://youtu.be/r9gyup4nzoQ?t=14 https://youtu.be/r9gyup4nzoQ?t=14 https://www.youtube.com/watch?v=44n_8oXJYD0 https://www.youtube.com/watch?v=44n_8oXJYD0
- WatchDog 5y ago
  Are there any products in this space using "smart routing" services like argo, or AWS global accelerator? I've seen really impressive latency improvements, just for HTTP based services, but this is the kind of use case where I would think they would really shine.
gamekathu 5y ago
“FarPlay has been successfully tested with upload speeds as low as 8 mbps.” that doesn’t sound low to me! Anyone tested this in poor 4G connections?
- Joyfield 5y ago
  8 millibits per second, sound a bit(!) low..... /s
- IshKebab 5y ago
  And the upload speed isn't the important thing anyway. How much latency and packet loss has it been tested with?
- zekica 5y ago
  You need 1.4Mbps for PCM 44100Hz 16bit Stereo. If they are using mono, 0.7Mbps.
mosselman 5y ago
What is the lowest latency voice chat application? Maybe for group calls?
- HoppyHaus 5y ago
  Mumble can get it very low. I don't know the actual stats, but audio packets can go to 10ms and use UDP
  - Wowfunhappy 5y ago
    Talking on mumble really feels amazing once you get it set up.
- hansel_der 5y ago
  mumble and teamspeak. big win here is that throu self-hosting one can pick the optimal place i.e. an isp "in the middle" or otherwise local to the participants. just stay away from places that host a lot of content (ovh, hetzner, big-cloud)
whistlegraph 5y ago
What I'd really love to see is this combined with low latency drawing / doodling. I wanna make pictured with friends fast enough to play games.
Darthy 5y ago
Related: sub.live, https://news.ycombinator.com/item?id=28968184 https://news.ycombinator.com/item?id=28968184
sirwitti 5y ago
I love the idea of connecting musicians over the internet. But in many music technology applications latency is critical and from a musician's standpoint the technically achievable latency is not suitable for actual music making apart from some exceptions - even with dedicated hardware. When making music that depends on a beat, pocket or groove any roundtrip latency larger than 3ms is noticeable and >6ms is not playable. If you play software synthesizers on your computer you probably will be aware of the issue since the sample rate and buffer settings of your soundcard alone introduce latency that ranges from playable (for me <4ms) to unacceptable. Since network latency alone can easily get worse than that I don't see technology like this being usable for playing serious music that focusses on rhythm. Note that the demo they use is piano and singer which in combination with the chosen song is decently forgiving latency-wise. I'd like to hear a demo with a grove where they switch between each participant to hear each musicians version.
- jokteur 5y ago
  As long as the latency is consistent, then it is possible to get used to it. Organists who play on big church organs frequently deal with huge latency, especially when playing with choirs or orchestras who are situated >30m away.
  - sirwitti 5y ago
    As it happens I used to play the organ and yes latency can be a real issue, especially in orchestras. Orchestras are notoriously behind conductors to the point that conductors conduct ahead of the time they want the beat to be. Drummers in orchestras need to guess (and have the experience) at which point after the conductor's beat they hit in order to be in sync with most of the other musicians. That's a tough spot to be in latency-wise :) Apart from that organ music tends not to have a strong focus on rhythm and groove, in which case latency is less of an issue.
snvzz 5y ago
Took a look. Highlight: Not open source and no mention of end to end encryption. Hard pass.
qeternity 5y ago
Interesting choice of name given that FairPlay was (is?) Apple's DRM scheme for many years. Ugh, I am feeling old.
- fabioborellini 5y ago
  is
- deleted 5y ago
  [deleted]
max_ 5y ago
Are there any benchmarking service for ranking audio latency of such apps?
chanux 5y ago
The video demo looks great to me. Conveys what farplay does really well.
ksec 5y ago
Are there any recent comparison of Audio I/O latency between different OS? Most I find were nearly 20 years old.
dahfizz 5y ago
> FarPlay is a state-of-the-art app for communicating in ultra-low-latency with others over the internet. I mean... its still doing a round trip over the internet, right? Maybe I just have a very different definition of "ultra-low-latency".
- willeh 5y ago
  Everything is relative and everything is a tradeoff, so I think what they are trying to say with ultra-low latency is that they have tried to minimise possible sources of latency and that make all possible trade-offs to favour lower latency. A streaming music service might for instance buffer audio (which incurs latency) to get reliability, which FarPlay probably doesn't do
- tomfanning 5y ago
  Yes but. I'm looking right now at a mtr (traceroute) screen from a rural fibre-to-the-premises broadband connection in Lancashire, north-west UK, achieving 2.5ms RTT to Manchester (the nearest major city, ~60km) and 8.5ms RTT to London (~325km). Things are improving. That's only ~10x and ~8x the light travel time, assuming straight lines which is best case. ~8.3x and ~5.3x respectively if you consider that light is much slower in fibre than air.
torginus 5y ago
What's the point of this? Stuff like Stadia already works on top of WebRTC, and they can have sub ~100ms latency, where most of it is in the network, which no amount of driver trickery can do anything about. Add to this the fact that humans are generally more tolerant of audio lag, than video, and I'm not sure if you don't have video, or real-time input, how do you event detect such low latencies. If it's for musicians, as depicted on the page, you generally need to connect more than 2 of them.
- dantepfer 5y ago
  Humans are way less tolerant of audio lag than video lag, in fact.
- sp332 5y ago
  It's different when a game is making a sound and you just hear it a bit later than you would if it were local. This case is for when you need a local sound to occur at the same time as a sound sent over the network. Latency is very noticeable.
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]

204 points by sp332 5y ago

Does this work between two devices on the same LAN? - edit - Thanks to another user I found: https://www.sonobus.net/ https://www.sonobus.net/ Which does seem to work over LAN.

Shared404 5y ago
I'm curious, what's the use case here? Not that it's wrong to do or anything, just intrigued.
- jeroenhd 5y ago
  Not the parent post, but I've been thinking about unifying all devices in my home by sending all audio streams into a single, unified "sound server" from which I can easily switch between TV audio, Bluetooth, headphones, etc. The idea is to be able to take a call on my phone and receive notifications on my laptop while developing on my PC. It'd also be nice for streaming stuff in general. As many people who have also looked into this stuff probably already know, interactive audio is really sensitive to latency. I've given it a quick shot by running pulse over the network, but honestly I wasn't impressed. I've given up for now, but some suggested software here might be good enough for a second shot, based on pipewire and its JACK backend this time.
  - getcrunk 5y ago
    Hear hear. I was just thinking about a putting all my music and a mpd on my pi then running clients on tv/computers/phones. But ur idea is the next step!
    - Shared404 5y ago
      > I was just thinking about a putting all my music and a mpd on my pi then running clients on tv/computers/phones I actually do something similar to this to run music for an online D&D campaign I'm in, except I use the http output so none of the other players have to install a client. I do like the idea of the home audio server use case, I think it'll go on my back burner!
  - smaudet 5y ago
    That sounds too connected - I like the idea of some sort pulseaudio network services but with better latency. I'd like to see pulseaudio with a better cross platform UI and a set of decentralized services versus introducing a single point of failure like you are proposing. Maybe a solution with dns-sd with stun support and a slick UI for desktop/mobile?
  - iostream23 5y ago
    Or AVB as I note above, depending upon your lan and hardware
- hunterb123 5y ago
  My wife and I play video games together on separate PCs. Our computers are beside each other so we need to use headsets, it's important to hear 360 audio for footsteps and other sounds in shooters. We need to hear each other, game audio, teammates voices, and each other's voice. Our voices can sometimes bleed through the headsets so we need there to be little to no lag to reduce reverb. Plus the faster our voices reach each other the faster we can react to the command / callout. We can't use voice channels in the game, discord, or anything that goes over the internet. Currently we use teamspeak with a local server, which works well, looking to reduce it more. Sometimes we simply play with one headphone ear partially off, but that ruins the surround sound.
  - Shared404 5y ago
    Ah, makes sense, thanks for satisfying my curiosity!
  - Ugohcet 5y ago
    You probably should try to do it old analog style: just plug your microphones into each other’s PCs and set up soundcard's mixer to add mic input into the mix
  - Moru 5y ago
    Did you try a local mumble server? Haven't played with it for a long time but last time it was much faster than teamspeak and we used it happily at home sitting next to eachother. That and get better headphones that close out sound from environment :-)
    - hunterb123 5y ago
      Mumble was a bit slower for us than TS3 actually. Around 40ms avg where TS3 is 15-30ms.
iostream23 5y ago
Well, there’s actually a standard for low latency LAN audio that uses PTP “precision timing protocol” a frame prioritization protocol. Given a switch that supports PTP, including some bog-standard Netgear models, you can use what is called AVB or audio video bridge. You can use an AVB driver directly on each system and audio being processed on one system can playback over another’s audio hardware! So, there’s also a layer three standard called AES67 from the Audio Enfineering Society and this competes directly with systems like Dante. I have no idea how these perform over the broader general internet as I would think that each hop alters the latency landscape significantly

dantepfer 5y ago

Hi everyone, Dan Tepfer here, co-creator of FarPlay. As a skeptic myself, I appreciate the skeptical tone of this thread. And I'd like to address a few points made here about FarPlay and low-latency audio. First, a little about me: I've been coding for most of my life (see https://www.youtube.com/watch?v=SaadsrHBygc https://www.youtube.com/watch?v=SaadsrHBygc for my NPR Tiny Desk Concert of my improvised algorithmic music project Natural Machines) but I'm first and foremost a musician. I make my living playing concerts around the world as a pianist. During the pandemic, I used JackTrip to perform remote livestream concerts with some of the greatest musicians in jazz: Christian McBride, Cécile McLorin Salvant, Ben Wendel, Gilad Hekselman, Fred Hersch, Antonio Sanchez, Melissa Aldana, Miguel Zenon, Linda May Han Oh and others. This is just to say that music, and particularly rhythm, is very important to me, and that I care about low-latency audio as an active practitioner. Someone in this thread wrote that for rhythmic (groove) music, latencies of 3ms are noticeable, and latencies higher than 6ms are prohibitive. This isn't the case. Sound travels in air at about 1ft per ms, so a latency of 3ms is equivalent to playing with someone 3ft away from you, which is obviously unnoticeable. 6ms is equivalent to playing with someone 6ft away, which is also unnoticeable. James Brown grooved his ass off with his band spread out over a relatively wide area on stage, long before in-ear monitors, which confirms what the research says: even for advanced professional musicians, latencies up to 20ms (equivalent to 20ft in air) are not significantly noticeable even for intricately rhythmic music. Here's an excerpt over JackTrip with Christian McBride, where at the end, we play a demanding bebop head in unison, a very tough test of latency: https://www.facebook.com/watch/?v=1076063889493342 https://www.facebook.com/watch/?v=1076063889493342. Above 20ms, things do get noticeable, but depending on the type of music you're playing, it's possible to adjust. It starts to feel like the people you're playing with are, as we say in the jazz world, "laying back on the beat". For example, I did a livestream performance for the French Institute in NYC last January with pianist Thomas Enhco in Paris and myself in Brooklyn, 3500 miles away, and despite a clearly noticeable (to us) ~40ms of latency, we were able to make real music together, including rhythmic music. Note that at the time, using JackTrip, I couldn't accurately estimate the actual latency, and this 40ms figure is a guess. FarPlay, in contrast, measures the current latency and displays it on the connection screen. Someone mentioned Jamulus. I've tried Jamulus, and for my professional needs, which include rock-solid stability and the lowest possible latency, JackTrip is far superior. But JackTrip, as someone else pointed out here, is impossible to use for the average user. It requires opening ports on your router, interacting with the command line, and installing and using Jack, which itself is forbidding for most users. Our goal with FarPlay was to take the best elements of JackTrip, unbeatable stability and latency, and make them easily accessible. SonoBus is also mentioned in this thread. SonoBus is an excellent project which we only came across a few months ago. We've tried it, and we've found that if you measure the actual sound-to-sound latency, i.e. the time from sound production at the source to sound reproduction at the destination, FarPlay achieves lower latencies than SonoBus, probably because of the way it processes audio internally. Also, we believe our interface, which we've put a lot of thought into, is easier to use for non-technical musicians than SonoBus. Another advantage of FarPlay over SonoBus, this one particularly important to me as a live performer, is Broadcast Output, which is an essential feature of JackTrip that FarPlay co-creator Anton Runov and I invented (see https://farplay.io/about#history https://farplay.io/about#history). To play in low latency, it's often necessary for the musicians to tolerate artifacts in the audio, since some audio packets inevitably get delayed on their way. Broadcast Output allows you to play in low latency with artifacts in your headphones, while simultaneously outputting artifact-free audio for live broadcast or recording. To me, this is the holy grail of remote performance, allowing us to have our cake and eat it too — ultra-low-latency interaction with no sacrifice in final audio quality (see https://farplay.io/tipsandtricks#broadcastoutput https://farplay.io/tipsandtricks#broadcastoutput). I should mention that FarPlay only allows one-to-one connections at the moment, while SonoBus allows multi-user sessions. We plan to add multi-user sessions to the FarPlay user interface soon; our underlying processes already allow them. Some of you are nitpicking our claim to not require third-party software. Remember, we're coming from JackTrip, which requires users to install and use Jack in addition to JackTrip. On Mac and Linux, there is no third-party software whatsoever required. On Windows, low-latency audio is currently impossible without ASIO drivers. Many musicians on Windows have audio interfaces with ASIO drivers already installed, so in their case there are no additional downloads required. If you don't have an ASIO driver, you'll have to use ASIO4ALL, but this true for any software doing low-latency audio on Windows. In essence, FarPlay is as self-contained as it can be at this stage. Someone asked if FarPlay will connect two users on the same LAN. The answer is yes, it works great. Someone else brought up the advantages of low-latency audio not only for music, but also for regular conversations. We wholeheartedly agree: conversations feel vastly more natural without the awkward delay added by Zoom, FaceTime, WhatsApp and regular phone calls. FarPlay also transmits uncompressed audio, so the quality is as good as your mic and sound card can provide, which also helps conversations feel more real. In conclusion, I want to thank you for bringing your attention to FarPlay, and if you enjoy playing music with other people, we'd love for you to try it! It's really quite magical, I feel, and the magic hasn't worn off for me even after having done it regularly for over a year. We've tried to make the process of using FarPlay as frictionless as possible: you don't even need to register for an account to use it, just download it (https://farplay.io/download https://farplay.io/download) and go. Thanks and Happy Thanksgiving to those of you who celebrate, --Dan Tepfer https://dantepfer.com https://dantepfer.com

brrrrrm 5y ago

The documentation say this is based on JackTrip, code here: https://github.com/jacktrip/jacktrip https://github.com/jacktrip/jacktrip

fredguth 5y ago
Jacktrip is really hard do set up in your non-tech musician friend's computer.
- sc20910 5y ago
  @fredguth - thats actually the reason for the MTA Easybox - a jacktrip appliance needing no knowledge to run at all. Contact networksound.com for info

ReactiveJelly 5y ago

> You don't need to install any third-party software Oh, so this is like a web browser thing? > Click here to download FarPlay Oh, so I _do_ need to install FarPlay. Just not any software that's a third party besides FarPlay. Which wouldn't make any sense.

tyingq 5y ago
>Just not any software that's a third party besides FarPlay. Which wouldn't make any sense. Heh. "Note for Windows: To use FarPlay on Windows, you must have an ASIO audio driver. We recommend the free ASIO4ALL."
- akiselev 5y ago
  Coming soon to a Windows automatic update near you, next time you're running a business critical overnight batch job.
- deleted 5y ago
  [deleted]
- smoldesu 5y ago
  Hope you're not planning to play with a backing track, ASIO4ALL notoriously does not play nice with multiple audio sources. It's almost like someone wanted to backport the horrors of ALSA to Windows because they missed how annoying it was having a single pair of inputs and outputs.
  - whyoh 5y ago
    >ASIO4ALL notoriously does not play nice with multiple audio sources. Yep, that's because Asio4all uses WDM-KS and WDM-KS since Vista doesn't support multiple sources. Actual ASIO drivers made by sound card manufacturers usually don't have this limitation, as long as you keep the same sample rate everywhere. But it can also vary depending on who made the driver and/or on whether the source apps are using ASIO or a mix of ASIO and WDM/WASAPI. Getting low latency audio to work nicely on Windows can be messy (compared to macOS, at least).

tonymet 5y ago

how can it support a p2p socket without opening ports ? i’m confused

sp332 5y ago
Stateful firewalls let UDP traffic out. And they generally let "replies" come back from the destination IP & port to the source IP and port on the host. That means that you just need a central server to coordinate the session ID and the IP & port of each side. Then the two peers can send each other data directly and the firewall will let it through because it's a "reply" to the data that was sent.
davidhyde 5y ago
I think they are referring to opening ports on your router as opposed to the automatic opening of incoming ports when connections are opened from the client machine. They probably use the STUN protocol to facilitate connecting two machines together peer-to-peer like what is done with WebRTC.
spacechild1 5y ago
See https://en.m.wikipedia.org/wiki/UDP_hole_punching https://en.m.wikipedia.org/wiki/UDP_hole_punching

gonesilent 5y ago

see also ninjam if you can't get over lag because of distance limitations.

adar 5y ago

I've been looking for a simple way to be able to listen to audio from my desktop and laptop simultaneously without requiring a hardware mixer or something. I'm currently using Scream but there is a bit of latency/delay. Farplay seems to be more for audio recording, but would Farplay be a good solution for my use case?

smegsicle 5y ago
shouldn't VLC make audio sync easy?
Kubuxu 5y ago
If you are on Windows and in a single network take a look at Voicemeeter and VBAN: https://vb-audio.com/Voicemeeter/banana.htm https://vb-audio.com/Voicemeeter/banana.htm
- adar 5y ago
  This is a good tip but unfortunately I'm on Windows on one computer and Fedora on the other.
dantepfer 5y ago
This isn't a use case that we've considered, but FarPlay would work well for this. If you want to play a file on one computer and have it play at the same time on the other, you'll have to play the file into FarPlay by using something like BlackHole, Jack or LoopBack, depending on your platform.

plusCubed 5y ago

What's the difference between this and Sonobus [1], which is open source and also supports iOS/Android? [1] https://www.sonobus.net/ https://www.sonobus.net/

FractalHQ 5y ago
Sonobus is incredible, it also ships with VST plugins for routing audio tracks from a DAW into a room.

deleted 5y ago

[deleted]

smoldesu 5y ago

Why does a peer-to-peer app like this need a subscription?

deleted 5y ago
[deleted]
discordance 5y ago
Generating those sessions to punch through NAT will need some sort of backend service, but I don't think it's subscription worthy. I would be happy to pay for this if it were a once off once the beta is over, and if FarPlay needs some dev money for upcoming features or functionality, then I would be happy to pay for the updates as well instead of a subscription.
- WatchDog 5y ago
  Mozilla used to run a free STUN service, but it looks like they killed it after it after it was abused to conduct certain attacks[0]. [0]: https://bugzilla.mozilla.org/show_bug.cgi?id=1324520 https://bugzilla.mozilla.org/show_bug.cgi?id=1324520
oliverevans96 5y ago
Not all clients have an available P2P route between them, and require a dedicated TURN server to forward the traffic between them. So there are some ongoing compute costs. But they also put the upfront effort into writing the code, and there is an ongoing maintenance burden. And they're looking for a way to turn a profit on their work.

rectang 5y ago

Oh good golly yes please we need innovation in this space. I'm so tired of having cell phone calls and video calls with long latencies. Latency is so disruptive to the normal flow of conversation. If there was a reliable low-latency alternative I would try to hold all my regular convos over it, starting with family calls and proceeding to work and professional calls if feasible.

agret 5y ago
Mumble is pretty great for low-latency audio. TeamSpeak3 too.
- Moru 5y ago
  Less great for setting up at your it-challenged friends though. And most people expect some phone call interface that beeps at them when they get a call.
- jagger27 5y ago
  When Discord first launched and my usual TeamSpeak friends moved over there I was super annoyed by the extra latency. How a group of fairly serious gamers who otherwise complained about less lag in any other circumstance shrugged it off, I'm not sure. More recently, I was surprised by low latency was between two Asterisk servers in the same city on different ISPs. It was a very adhoc setup with a cheap EOL Cisco IP phone on either end connected to an Asterisk SIP server on the local network. I'm so used to laggy voice chat that it actually caught me off guard how nice it was once I actually got it to work. Besides being a total nightmare to configure. I struggle enough in person to find the right time to talk without interrupting, and >100ms of Discord latency makes it that much worse for me. I hope something peer-to-peer like this catches on for remote teams. I could really benefit from it. I don't think I would mind if a screen share lagged behind the speaker's voice. I'm sure Zoom already does that though.
  - WolfRazu 5y ago
    You should be getting considerably less latency than that. Is the Discord server perhaps set to the wrong region?

raggi 5y ago

Preface: the product is probably fine, and certainly provides a nice simple UI which is perfect for it's intended use cases. There is something fishy about the video, though. You see in the video that Zoom has been keeping audio & video in sync. In the first latency demo (https://youtu.be/Zju8YaRcSk4?t=50 https://youtu.be/Zju8YaRcSk4?t=50), we see not only latency on the audio, but also latency on the video. In the second latency demo (https://youtu.be/Zju8YaRcSk4?t=225 https://youtu.be/Zju8YaRcSk4?t=225), once FarPlay is handling audio and they've muted Zoom, magically they've managed to synchronize the video. Was this done post-edit for some reason?

axiosgunnar 5y ago
Yes, it's displayed in the video (as a text overlay) as an info text
- raggi 5y ago
  Oh so it does (https://youtu.be/Zju8YaRcSk4?t=217 https://youtu.be/Zju8YaRcSk4?t=217), I missed it :)

axiosgunnar 5y ago

Would using something like this improve the quality of typical business meeting calls? E.g. reduce people talking over each other, reduce mental stress caused by the delay? Compared to Whatsapp/Zoom/etc

sp332 5y ago
It currently only supports two nodes connecting. The older jacktrip code supports more nodes, and FarPlay says this feature is coming later. But yeah, this would help a lot I think.

rdtennent 5y ago

"No dedicated hardware"; but for best results you'll need a wired ethernet connection, wired headphones, a good microphone and an audio interface, and on Windows an ASIO driver. "For the lowest possible latency, FarPlay establishes peer-to-peer connections between users": Except when the peers are on different ISP networks and the packets are forced to travel to some distant node, introducing latency at every node along the way. In my experience with this issue using Jamulus, having a server in the cloud at a suitable location reduces latency. Minimizing "the distance travelled" isn't important if there are few nodes introducing latency. The speed of light along wires is orders of magnitude faster than the speed of sound through air. "the faster your connection, the better your results will be"; Misleading; it's latency not bandwidth that's critical and a "fast" connection normally refers to bandwidth. Anyone interested in audio over the internet should check out Jamulus at https://jamulus.io/ https://jamulus.io/

tekstar 5y ago
Agree on jamulus. I re-joined my highschool band during the pandemic, despite us all living in different cities. Jamulus jams were an important part of us getting back together, along with zoom calls and passing recordings back and forth. Best thing that happened for me out of the pandemic.
- gavinray 5y ago
  If anyone is looking for realtime collaboration tools, there's a free (proprietary) one that has live video + audio chat and a shared project workspace for building songs. https://beatconnect.com/ https://beatconnect.com/ The homepage does a pretty poor job of showing what it is, so it'll probably make more sense if you just see it: https://youtu.be/r9gyup4nzoQ?t=14 https://youtu.be/r9gyup4nzoQ?t=14 https://www.youtube.com/watch?v=44n_8oXJYD0 https://www.youtube.com/watch?v=44n_8oXJYD0
WatchDog 5y ago
Are there any products in this space using "smart routing" services like argo, or AWS global accelerator? I've seen really impressive latency improvements, just for HTTP based services, but this is the kind of use case where I would think they would really shine.

gamekathu 5y ago

“FarPlay has been successfully tested with upload speeds as low as 8 mbps.” that doesn’t sound low to me! Anyone tested this in poor 4G connections?

Joyfield 5y ago
8 millibits per second, sound a bit(!) low..... /s
IshKebab 5y ago
And the upload speed isn't the important thing anyway. How much latency and packet loss has it been tested with?
zekica 5y ago
You need 1.4Mbps for PCM 44100Hz 16bit Stereo. If they are using mono, 0.7Mbps.

mosselman 5y ago

What is the lowest latency voice chat application? Maybe for group calls?

HoppyHaus 5y ago
Mumble can get it very low. I don't know the actual stats, but audio packets can go to 10ms and use UDP
- Wowfunhappy 5y ago
  Talking on mumble really feels amazing once you get it set up.
hansel_der 5y ago
mumble and teamspeak. big win here is that throu self-hosting one can pick the optimal place i.e. an isp "in the middle" or otherwise local to the participants. just stay away from places that host a lot of content (ovh, hetzner, big-cloud)

whistlegraph 5y ago

What I'd really love to see is this combined with low latency drawing / doodling. I wanna make pictured with friends fast enough to play games.

Darthy 5y ago

Related: sub.live, https://news.ycombinator.com/item?id=28968184 https://news.ycombinator.com/item?id=28968184

sirwitti 5y ago

I love the idea of connecting musicians over the internet. But in many music technology applications latency is critical and from a musician's standpoint the technically achievable latency is not suitable for actual music making apart from some exceptions - even with dedicated hardware. When making music that depends on a beat, pocket or groove any roundtrip latency larger than 3ms is noticeable and >6ms is not playable. If you play software synthesizers on your computer you probably will be aware of the issue since the sample rate and buffer settings of your soundcard alone introduce latency that ranges from playable (for me <4ms) to unacceptable. Since network latency alone can easily get worse than that I don't see technology like this being usable for playing serious music that focusses on rhythm. Note that the demo they use is piano and singer which in combination with the chosen song is decently forgiving latency-wise. I'd like to hear a demo with a grove where they switch between each participant to hear each musicians version.

jokteur 5y ago
As long as the latency is consistent, then it is possible to get used to it. Organists who play on big church organs frequently deal with huge latency, especially when playing with choirs or orchestras who are situated >30m away.
- sirwitti 5y ago
  As it happens I used to play the organ and yes latency can be a real issue, especially in orchestras. Orchestras are notoriously behind conductors to the point that conductors conduct ahead of the time they want the beat to be. Drummers in orchestras need to guess (and have the experience) at which point after the conductor's beat they hit in order to be in sync with most of the other musicians. That's a tough spot to be in latency-wise :) Apart from that organ music tends not to have a strong focus on rhythm and groove, in which case latency is less of an issue.

snvzz 5y ago

Took a look. Highlight: Not open source and no mention of end to end encryption. Hard pass.

qeternity 5y ago

Interesting choice of name given that FairPlay was (is?) Apple's DRM scheme for many years. Ugh, I am feeling old.

fabioborellini 5y ago
is
deleted 5y ago
[deleted]

max_ 5y ago

Are there any benchmarking service for ranking audio latency of such apps?

chanux 5y ago

The video demo looks great to me. Conveys what farplay does really well.

ksec 5y ago

Are there any recent comparison of Audio I/O latency between different OS? Most I find were nearly 20 years old.

dahfizz 5y ago

> FarPlay is a state-of-the-art app for communicating in ultra-low-latency with others over the internet. I mean... its still doing a round trip over the internet, right? Maybe I just have a very different definition of "ultra-low-latency".

willeh 5y ago
Everything is relative and everything is a tradeoff, so I think what they are trying to say with ultra-low latency is that they have tried to minimise possible sources of latency and that make all possible trade-offs to favour lower latency. A streaming music service might for instance buffer audio (which incurs latency) to get reliability, which FarPlay probably doesn't do
tomfanning 5y ago
Yes but. I'm looking right now at a mtr (traceroute) screen from a rural fibre-to-the-premises broadband connection in Lancashire, north-west UK, achieving 2.5ms RTT to Manchester (the nearest major city, ~60km) and 8.5ms RTT to London (~325km). Things are improving. That's only ~10x and ~8x the light travel time, assuming straight lines which is best case. ~8.3x and ~5.3x respectively if you consider that light is much slower in fibre than air.

torginus 5y ago

What's the point of this? Stuff like Stadia already works on top of WebRTC, and they can have sub ~100ms latency, where most of it is in the network, which no amount of driver trickery can do anything about. Add to this the fact that humans are generally more tolerant of audio lag, than video, and I'm not sure if you don't have video, or real-time input, how do you event detect such low latencies. If it's for musicians, as depicted on the page, you generally need to connect more than 2 of them.

dantepfer 5y ago
Humans are way less tolerant of audio lag than video lag, in fact.
sp332 5y ago
It's different when a game is making a sound and you just hear it a bit later than you would if it were local. This case is for when you need a local sound to occur at the same time as a sound sent over the network. Latency is very noticeable.

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]