7 ms·

A Badass Way to Connect Programs Together (2016)

84 points by pizza 5y ago

eternityforest 5y ago
OSC is cool, but I'd say MIDI is often better if networks aren't being used, simply because the routing and connections can be centrally managed with JACK or PipeWire or the like. If networks are being used, MQTT is my method of choice. It is so much easier to develop and debug when everything is in one place and you have a broker. Even if there's no network needed, I'd much rather see something embed an MQTT broker than a plain OSC socket, because reliability is often desirable, you often want to know if you are disconnected, etc. MQTT gives you a lot of that stuff for free. Plus, it gives you retained messages and last will and testament support. None of these are really perfect for all use cases though. I wish we had something like MQTT but for streaming media.
- joezydeco 5y ago
  I've been using an internal MQTT broker as a loose IPC bus on my latest embedded Linux project and it's been awesome. I'm never using DBus again. The revelation was when I had a developer overseas that needed to test something and didn't have hardware. One firewall change and an exposed port and now he was talking to a live target from halfway around the world.
  - eternityforest 5y ago
    I love DBus too, but it's definitely not the best embedded tool. Would have kinda been nice if a variant of MQTT had been the standard Linux broker for everything, but DBus does give some cool extras. There's almost nothing that you actually can't do nicely in MQTT though. It's a fantastic protocol.
    - joezydeco 5y ago
      The only place it kind of falls down as an IPC is confirmation that a specific subscriber saw a message you published. That's fixable with a little bit of shim code to send back an ACK, and that's something I can live with. The protocol wasn't really designed to be a round-trip communication system.
- sriku 5y ago
  Any comments on observed latencies and jitter when using MQTT? Musical applications are quite demanding on that front.
  - eternityforest 5y ago
    Latency can be good, or fairly bad, largely depending on whether you have Nagle's algorithm turned on or not.
- mxmilkiib 5y ago
  JACK (and, hence, PipeWire) also has CV, for a better resolution than MIDI.
  - PaulDavisThe1st 5y ago
    CV has zero semantics, and so it can't really be compared to MIDI. What a given CV "signal" does depends on its destination as much as its source, and in that respect, is somewhat similar to OSC, despite being more "universal" yet simultaneously "semantic free".
    - eternityforest 5y ago
      Semantics seem to somerimes get in the way unless they are fully specified into well defined profiles. Which is amazing when they are, in the form of "Any port of this type can meaningfully connect to any other port and it does what you think it should". But a lot of the time you wind up with something that STILL needs config on both ends.... because programmers hate arbitrary choices and always like to leave room for omitting features or simplifying later. And of course, programmers also like to break things, so even if semantics DO work, they won't hesitate to make v2. As much as I really love modular pluggable systems, it seems like semantics free things are unfortunately easier to build an ecosystem around. Nobody gets excited for a chart of data types it seems. OSC has survived surprisingly well though.
    - mxmilkiib 5y ago
      You can assign a CC to control a value, or you can use CV to control a value, the latter gives more resolution, so one can compare them.
      - PaulDavisThe1st 5y ago
        Many of the MIDI CC's have defined semantics, and another bunch do not. Many MIDI receivers choose to ignore those defined semantics (e.g. bind "volume control" messages to some arbitrary parameter). If you choose to ignore the semantics for the defined CC's, or only use the undefined ones, then MIDI CC does become more like CV. However, that's not the intention of the MIDI specification, and it certainly doesn't cover the rest of the MIDI specification, which dwarfs the CC part.
- CalRobert 5y ago
  Randomly, I always thought it was cool early force feedback joysticks used midi for bidirectional data in the game port.
jmpz 5y ago
https://opensoundcontrol.stanford.edu/ https://opensoundcontrol.stanford.edu/ is a good resource to get started with OSC. They have a nice list of implementations you can check out : https://opensoundcontrol.stanford.edu/page-list.html#implementations https://opensoundcontrol.stanford.edu/page-list.html#impleme... It might also be interesting to check out : https://cnmat.berkeley.edu/ https://cnmat.berkeley.edu/. OSC was developed at CNMAT (http://www.cnmat.berkeley.edu/OpenSoundControl/publications/wright-organisedsound-OSC.pdf http://www.cnmat.berkeley.edu/OpenSoundControl/publications/...) to facilitating musical networking.
dang 5y ago
One past thread: A Badass Way to Connect Programs Together - https://news.ycombinator.com/item?id=10976737 https://news.ycombinator.com/item?id=10976737 - Jan 2016 (145 comments)
claytongulick 5y ago
OSC is very cool, but I disagree with the author's negativity towards JSON. Different wire protocols are appropriate for different problem domains. I shudder to think of the monstrous API spec that would result from trying to do something like, for example, FHIR with OSC. There are many problem domains where readability is more important than wire efficiency.
- dnautics 5y ago
  I love Joe but he was wrong about JSON in exactly the way you describe.
- pizza 5y ago
  You may find this interesting (at least once it gets a little more developed again): https://github.com/wires/wtf https://github.com/wires/wtf type-checked JSON w/ custom sum and product types
PaulDavisThe1st 5y ago
Two problems with OSC: 1. There are no standard semantics for the messages. With MIDI, if you want to tell something that makes noise to start making noise, you send it a NoteOn message. While that may or may not do precisely what you want, the semantics are clear. With OSC, no such message exists. Every OSC receiver defines its own set of messages with their own semantics. 2. String parsing. It really does cost quite a lot to parse and dispatch OSC, because everything is string based. You can argue that it's worth it because of the flexibility, and in many cases, that's probably the right verdict. It doesn't get rid of the cost, however.
uwagar 5y ago
anybody connect programs using 'shared memory'? guess there is a POSIX compliant way to do it cross platform?
- GianFabien 5y ago
  Unix domain sockets are comparably performant to shared memory without the need for semaphores and locks. Of course, it only works on single node deployments. UDP/TCP based protocols are usable on multi-node deployments.
  - uwagar 5y ago
    indeed, also when a lot of data needs to be transfered with low latency between 2 programs on the same computer, shared memory method could be more attractive. can u transmit a huge block of data in OSC in one go? or is it attribute, value pair?
varjag 5y ago
TCP uses the idea a session and the only rational way to program a session is as a process or (horrors) as a thread. Ugh… events/poll?
- GianFabien 5y ago
  I recently migrated some services to uasyncio with Python. I was most impressed by how much code I was able to delete compared to my old implementation. I am yet to try the same approach with Deno (the next generation Node.JS).
radarsat1 5y ago
Might as well plug something here, forgive me: Years ago I co-developed a C library to connect OSC endpoints together in a decentralized manner and semantically translate messages with arbitrary operations such a scaling and filtering. There are still a few users and developers. http://libmapper.github.io/about.html http://libmapper.github.io/about.html
- eternityforest 5y ago
  That's amazing! Way more things should use this communication model! Auto peer discovery is great.
  - radarsat1 5y ago
    Thanks! One feature that we developed, called "instances", is actually quite interesting, we ended up even publishing paper just on that topic [1]. It's a proposed way to design mappings that handles the idea of signals that "appear and disappear", or might have multiple instances, like in the context of a multitouch display. So instead of mapping a specific signal (eg. position of an active touch), you map a signal "class" which can be "instantiated", and the receiving side can automatically handle this, e.g. a synthesizer can allocate or deallocate voices in response. The protocol handles all events and corner cases such as the two sides having asymmetric resources, or the associated instances along the chain having asymmetric lifetimes, etc. This is one thing that we feel is quite unique to this solution and does something that MIDI, or other protocols that we know of, really cannot do (well). [1] https://www.researchgate.net/profile/Stephen-Sinclair-4/publication/320841558_Generalized_Multi-Instance_Control_Mapping_for_Interactive_Media_Systems/links/5e06006ea6fdcc2837429fac/Generalized-Multi-Instance-Control-Mapping-for-Interactive-Media-Systems.pdf https://www.researchgate.net/profile/Stephen-Sinclair-4/publ...

84 points by pizza 5y ago

OSC is cool, but I'd say MIDI is often better if networks aren't being used, simply because the routing and connections can be centrally managed with JACK or PipeWire or the like. If networks are being used, MQTT is my method of choice. It is so much easier to develop and debug when everything is in one place and you have a broker. Even if there's no network needed, I'd much rather see something embed an MQTT broker than a plain OSC socket, because reliability is often desirable, you often want to know if you are disconnected, etc. MQTT gives you a lot of that stuff for free. Plus, it gives you retained messages and last will and testament support. None of these are really perfect for all use cases though. I wish we had something like MQTT but for streaming media.

joezydeco 5y ago
I've been using an internal MQTT broker as a loose IPC bus on my latest embedded Linux project and it's been awesome. I'm never using DBus again. The revelation was when I had a developer overseas that needed to test something and didn't have hardware. One firewall change and an exposed port and now he was talking to a live target from halfway around the world.
- eternityforest 5y ago
  I love DBus too, but it's definitely not the best embedded tool. Would have kinda been nice if a variant of MQTT had been the standard Linux broker for everything, but DBus does give some cool extras. There's almost nothing that you actually can't do nicely in MQTT though. It's a fantastic protocol.
  - joezydeco 5y ago
    The only place it kind of falls down as an IPC is confirmation that a specific subscriber saw a message you published. That's fixable with a little bit of shim code to send back an ACK, and that's something I can live with. The protocol wasn't really designed to be a round-trip communication system.
sriku 5y ago
Any comments on observed latencies and jitter when using MQTT? Musical applications are quite demanding on that front.
- eternityforest 5y ago
  Latency can be good, or fairly bad, largely depending on whether you have Nagle's algorithm turned on or not.
mxmilkiib 5y ago
JACK (and, hence, PipeWire) also has CV, for a better resolution than MIDI.
- PaulDavisThe1st 5y ago
  CV has zero semantics, and so it can't really be compared to MIDI. What a given CV "signal" does depends on its destination as much as its source, and in that respect, is somewhat similar to OSC, despite being more "universal" yet simultaneously "semantic free".
  - eternityforest 5y ago
    Semantics seem to somerimes get in the way unless they are fully specified into well defined profiles. Which is amazing when they are, in the form of "Any port of this type can meaningfully connect to any other port and it does what you think it should". But a lot of the time you wind up with something that STILL needs config on both ends.... because programmers hate arbitrary choices and always like to leave room for omitting features or simplifying later. And of course, programmers also like to break things, so even if semantics DO work, they won't hesitate to make v2. As much as I really love modular pluggable systems, it seems like semantics free things are unfortunately easier to build an ecosystem around. Nobody gets excited for a chart of data types it seems. OSC has survived surprisingly well though.
  - mxmilkiib 5y ago
    You can assign a CC to control a value, or you can use CV to control a value, the latter gives more resolution, so one can compare them.
    - PaulDavisThe1st 5y ago
      Many of the MIDI CC's have defined semantics, and another bunch do not. Many MIDI receivers choose to ignore those defined semantics (e.g. bind "volume control" messages to some arbitrary parameter). If you choose to ignore the semantics for the defined CC's, or only use the undefined ones, then MIDI CC does become more like CV. However, that's not the intention of the MIDI specification, and it certainly doesn't cover the rest of the MIDI specification, which dwarfs the CC part.
CalRobert 5y ago
Randomly, I always thought it was cool early force feedback joysticks used midi for bidirectional data in the game port.

jmpz 5y ago

https://opensoundcontrol.stanford.edu/ https://opensoundcontrol.stanford.edu/ is a good resource to get started with OSC. They have a nice list of implementations you can check out : https://opensoundcontrol.stanford.edu/page-list.html#implementations https://opensoundcontrol.stanford.edu/page-list.html#impleme... It might also be interesting to check out : https://cnmat.berkeley.edu/ https://cnmat.berkeley.edu/. OSC was developed at CNMAT (http://www.cnmat.berkeley.edu/OpenSoundControl/publications/wright-organisedsound-OSC.pdf http://www.cnmat.berkeley.edu/OpenSoundControl/publications/...) to facilitating musical networking.

dang 5y ago

One past thread: A Badass Way to Connect Programs Together - https://news.ycombinator.com/item?id=10976737 https://news.ycombinator.com/item?id=10976737 - Jan 2016 (145 comments)

claytongulick 5y ago

OSC is very cool, but I disagree with the author's negativity towards JSON. Different wire protocols are appropriate for different problem domains. I shudder to think of the monstrous API spec that would result from trying to do something like, for example, FHIR with OSC. There are many problem domains where readability is more important than wire efficiency.

dnautics 5y ago
I love Joe but he was wrong about JSON in exactly the way you describe.
pizza 5y ago
You may find this interesting (at least once it gets a little more developed again): https://github.com/wires/wtf https://github.com/wires/wtf type-checked JSON w/ custom sum and product types

PaulDavisThe1st 5y ago

Two problems with OSC: 1. There are no standard semantics for the messages. With MIDI, if you want to tell something that makes noise to start making noise, you send it a NoteOn message. While that may or may not do precisely what you want, the semantics are clear. With OSC, no such message exists. Every OSC receiver defines its own set of messages with their own semantics. 2. String parsing. It really does cost quite a lot to parse and dispatch OSC, because everything is string based. You can argue that it's worth it because of the flexibility, and in many cases, that's probably the right verdict. It doesn't get rid of the cost, however.

uwagar 5y ago

anybody connect programs using 'shared memory'? guess there is a POSIX compliant way to do it cross platform?

GianFabien 5y ago
Unix domain sockets are comparably performant to shared memory without the need for semaphores and locks. Of course, it only works on single node deployments. UDP/TCP based protocols are usable on multi-node deployments.
- uwagar 5y ago
  indeed, also when a lot of data needs to be transfered with low latency between 2 programs on the same computer, shared memory method could be more attractive. can u transmit a huge block of data in OSC in one go? or is it attribute, value pair?

varjag 5y ago

TCP uses the idea a session and the only rational way to program a session is as a process or (horrors) as a thread. Ugh… events/poll?

GianFabien 5y ago
I recently migrated some services to uasyncio with Python. I was most impressed by how much code I was able to delete compared to my old implementation. I am yet to try the same approach with Deno (the next generation Node.JS).

radarsat1 5y ago

Might as well plug something here, forgive me: Years ago I co-developed a C library to connect OSC endpoints together in a decentralized manner and semantically translate messages with arbitrary operations such a scaling and filtering. There are still a few users and developers. http://libmapper.github.io/about.html http://libmapper.github.io/about.html

eternityforest 5y ago
That's amazing! Way more things should use this communication model! Auto peer discovery is great.
- radarsat1 5y ago
  Thanks! One feature that we developed, called "instances", is actually quite interesting, we ended up even publishing paper just on that topic [1]. It's a proposed way to design mappings that handles the idea of signals that "appear and disappear", or might have multiple instances, like in the context of a multitouch display. So instead of mapping a specific signal (eg. position of an active touch), you map a signal "class" which can be "instantiated", and the receiving side can automatically handle this, e.g. a synthesizer can allocate or deallocate voices in response. The protocol handles all events and corner cases such as the two sides having asymmetric resources, or the associated instances along the chain having asymmetric lifetimes, etc. This is one thing that we feel is quite unique to this solution and does something that MIDI, or other protocols that we know of, really cannot do (well). [1] https://www.researchgate.net/profile/Stephen-Sinclair-4/publication/320841558_Generalized_Multi-Instance_Control_Mapping_for_Interactive_Media_Systems/links/5e06006ea6fdcc2837429fac/Generalized-Multi-Instance-Control-Mapping-for-Interactive-Media-Systems.pdf https://www.researchgate.net/profile/Stephen-Sinclair-4/publ...