5 ms·

I wish they would have removed SysEx messages. They cause way more trouble than they're worth, now that we have property exchange/profile configuration built in

I wish they would have removed SysEx messages. They cause way more trouble than they're worth, now that we have property exchange/profile configuration built into the spec.

dkersten 7y ago
What's the problem that SysEx messages cause? I've only limited experience with them (having used them only to send config data to a device, in a way that didn't need the device to keep functioning while being updated, from a programming point of view, I found them quite convenient and simple, except perhaps the fact that you only get 7 bits per byte so may have to pack the data).
- SeanLuke 7y ago
  [For the benefit of HN: SysEx is a special manufacturer-specific MIDI message which is undefined, so a synth manufacturer can use it for whatever he wishes] I build a lot of open source patch editors for older synthesizers. My beef with Sysex is that every manufacturer uses completely different approaches to defining their own proprietary messages with it. For example, nearly every synth in the universe has a sysex message for dumping a patch (a full set of parameter settings) from a synth to another or to a computer; but they define their messages is radically different ways, so I must construct an entirely different set of parsing and emitting tools for every single synthesizer, even within a given manufacturer. It's a nightmare. So continuing this example, if the MIDI association had gotten together early on and said that MIDI dumps should have a header that looks like THIS and then all the parameters in order, two bytes per parameter with no bit packing whatsoever, no two's complement, and end with a specific checksum, then I'd have written 10x more patch editors so far. I wouldn't have to write custom parsers and dumpers: I'd just provide a list of parameters and their bounds.
  - fit2rule 7y ago
    (Disclaimer: been working in synth industry for decades now..) Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct. So, it wouldn't really make much sense to have them all use the same struct - this can't be enforced too well. Forcing synth mfr's to all use the same struct means that, even if they have their own internal plain-old-structs, they'd need code to dump the SYSEX according to the standard.
    - SeanLuke 7y ago
      > Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct. I don't think it's processing power: it's stingy RAM utilization. Many bad actors (Kawai, Casio, later Yamaha) did crazy bit-packing of parameters rather than just keep them in a simple array, while the more sane (Oberheim, E-mu, Waldorf, early Yamaha) at least tried to pack in a consistent way. Other bad actors (ahem Korg, as late as 2000) decided to use, shall we say, creative parameter encodings, going even so far as embedding textified versions of parameter numbers into sysex byte streams. And many used all sorts of crazy schemes for banks and patch numbering, most of which are incompatible with one another. And it's not just encoding: basic synth patch dump features are missing from different models. There are five basic tasks that most synth editors require: - Change patch - Request a dump from working memory - Dump to working memory - Dump to patch RAM and save - Send a single parameter change (for any parameter) Manufacturers couldn't even agree to make machines which supported all five of these. Some machines (Yamaha) have no way to write to RAM. Some machines couldn't do single parameter changes. Some machines can't properly change patches in a consistent manner. Some machines have no patch request mechanism. Many machines can't dump to current working memory: only to patch RAM! The situation is only getting worse. Whereas in the past manufacturers at least attempted a complement of sysex messages, now many manufacturers can't even be bothered to allow access to their machines (Korg, Roland). Others treat their sysex messages as proprietary secrets (Arturia, Digitech, Alesis). There is only one truly good, shining actor in the open MIDI spec space, and that is Sequential. Which shouldn't be a surprise given who runs it.
      - PaulDavisThe1st 7y ago
        "Send a single parameter change (for any parameter)" This makes no sense. That would also imply a way to discover (and name, and probably provide semantics for) all parameters. That's a huge ask if MIDI (even MIDI 2.0) is the only communication protocol available. Yes, the first 4 of your list are common. The first one is covered by the core MIDI spec. The 2nd and 3rd have no standard msg, but your complaint seems to be about the contents of the message, which is no business of the requestor. The 4th assumes "patch RAM", which cannot be assumed, as you note, and that seems correct to me.
- unlinked_dll 7y ago
  I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message. This isn't uncommon in serial protocols, but MIDI has been lifted several layers above the UART it was designed for, and one of its strengths for everything but SysEx is that you exactly how big messages are going to be and preallocating space for them is trivial.
  - aikah 7y ago
    > I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message. That's the reason why you want to break compatibility with MIDI 1 spec? Really? Just like for an HTTP stream, you don't need to know how big the stream is to process it, just when it ends, there is no new problem to solve here.
    - unlinked_dll 7y ago
      No, I want there to be only one way to do something in the protocol and to remove arbitrary binary exchange, this is one pain point. It turns your "standard" into a Frankenstein's Monster of data exchange, and you'd think we'd have learned over the last few decades that ambiguity in specification of a communication protocol is just going to lead to headaches and incompatible/buggy hardware. It's also weird to say SysEx is needed for backwards compat with MIDI 1.0. You can just require a translation layer between MIDI 1.0 sysex and MIDI 2.0 configuration/property exchange for backwards compat. Opaque binary as a part of the protocol does not encourage compatibility among hardware.
      - aikah 7y ago
        The MIDI spec was successful for 30+ years for a reason and manufacturers managed with SysEx without any issue, there is no justification for any "compatibility layer" whatsoever here. The problem wasn't the spec obviously.
        7y ago
- uhoh-itsmaciek 7y ago
  It's been kind of a headache in Firefox's WebMIDI support plans (since SysEx is sometimes used for firmware updates and it's hard to explain "this page may hack your keyboard" in a permission request dialog): https://github.com/mozilla/standards-positions/issues/58#issuecomment-369067271 https://github.com/mozilla/standards-positions/issues/58#iss...
deleted 7y ago
[deleted]
8bitsrule 7y ago
Lots of 1.0 instruments have only SysEx to load/save patches. A single patch may have hundreds of parameters. Whole libraries of patches exist; they can be loaded/saved at once with Sysex. So, no way.
- unlinked_dll 7y ago
  I don't really see why that matters for a new protocol that fallbacks to MIDI 1.0, it's not a superset of MIDI 1.0. If your device only takes MIDI 1.0 SysEx for patches, anything that would send those patches via MIDI is going to do it on top of the fallback. What I'm saying is a new MIDI 2.0 device probably shouldn't be able to use SysEx. Either opt out of MIDI 1.0 and use the paradigms established or fallback to MIDI 1.0 and its limitations. Otherwise we're just going to get a mess of different implementations, and MIDI 2.0 will fail to be a successful standard.

I wish they would have removed SysEx messages. They cause way more trouble than they're worth, now that we have property exchange/profile configuration built into the spec.

dkersten 7y ago

What's the problem that SysEx messages cause? I've only limited experience with them (having used them only to send config data to a device, in a way that didn't need the device to keep functioning while being updated, from a programming point of view, I found them quite convenient and simple, except perhaps the fact that you only get 7 bits per byte so may have to pack the data).

SeanLuke 7y ago
[For the benefit of HN: SysEx is a special manufacturer-specific MIDI message which is undefined, so a synth manufacturer can use it for whatever he wishes] I build a lot of open source patch editors for older synthesizers. My beef with Sysex is that every manufacturer uses completely different approaches to defining their own proprietary messages with it. For example, nearly every synth in the universe has a sysex message for dumping a patch (a full set of parameter settings) from a synth to another or to a computer; but they define their messages is radically different ways, so I must construct an entirely different set of parsing and emitting tools for every single synthesizer, even within a given manufacturer. It's a nightmare. So continuing this example, if the MIDI association had gotten together early on and said that MIDI dumps should have a header that looks like THIS and then all the parameters in order, two bytes per parameter with no bit packing whatsoever, no two's complement, and end with a specific checksum, then I'd have written 10x more patch editors so far. I wouldn't have to write custom parsers and dumpers: I'd just provide a list of parameters and their bounds.
- fit2rule 7y ago
  (Disclaimer: been working in synth industry for decades now..) Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct. So, it wouldn't really make much sense to have them all use the same struct - this can't be enforced too well. Forcing synth mfr's to all use the same struct means that, even if they have their own internal plain-old-structs, they'd need code to dump the SYSEX according to the standard.
  - SeanLuke 7y ago
    > Most SYSEX dumps are just dumps of the plain ol' structs that the synth engines are using to drive their output. A lot of synths don't have the processing power to do more than just dump the struct. I don't think it's processing power: it's stingy RAM utilization. Many bad actors (Kawai, Casio, later Yamaha) did crazy bit-packing of parameters rather than just keep them in a simple array, while the more sane (Oberheim, E-mu, Waldorf, early Yamaha) at least tried to pack in a consistent way. Other bad actors (ahem Korg, as late as 2000) decided to use, shall we say, creative parameter encodings, going even so far as embedding textified versions of parameter numbers into sysex byte streams. And many used all sorts of crazy schemes for banks and patch numbering, most of which are incompatible with one another. And it's not just encoding: basic synth patch dump features are missing from different models. There are five basic tasks that most synth editors require: - Change patch - Request a dump from working memory - Dump to working memory - Dump to patch RAM and save - Send a single parameter change (for any parameter) Manufacturers couldn't even agree to make machines which supported all five of these. Some machines (Yamaha) have no way to write to RAM. Some machines couldn't do single parameter changes. Some machines can't properly change patches in a consistent manner. Some machines have no patch request mechanism. Many machines can't dump to current working memory: only to patch RAM! The situation is only getting worse. Whereas in the past manufacturers at least attempted a complement of sysex messages, now many manufacturers can't even be bothered to allow access to their machines (Korg, Roland). Others treat their sysex messages as proprietary secrets (Arturia, Digitech, Alesis). There is only one truly good, shining actor in the open MIDI spec space, and that is Sequential. Which shouldn't be a surprise given who runs it.
    - PaulDavisThe1st 7y ago
      "Send a single parameter change (for any parameter)" This makes no sense. That would also imply a way to discover (and name, and probably provide semantics for) all parameters. That's a huge ask if MIDI (even MIDI 2.0) is the only communication protocol available. Yes, the first 4 of your list are common. The first one is covered by the core MIDI spec. The 2nd and 3rd have no standard msg, but your complaint seems to be about the contents of the message, which is no business of the requestor. The 4th assumes "patch RAM", which cannot be assumed, as you note, and that seems correct to me.
unlinked_dll 7y ago
I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message. This isn't uncommon in serial protocols, but MIDI has been lifted several layers above the UART it was designed for, and one of its strengths for everything but SysEx is that you exactly how big messages are going to be and preallocating space for them is trivial.
- aikah 7y ago
  > I can't know how big a SysEx message is until run time, which makes buffering them somewhat complicated when you don't know what's in the SysEx message. That's the reason why you want to break compatibility with MIDI 1 spec? Really? Just like for an HTTP stream, you don't need to know how big the stream is to process it, just when it ends, there is no new problem to solve here.
  - unlinked_dll 7y ago
    No, I want there to be only one way to do something in the protocol and to remove arbitrary binary exchange, this is one pain point. It turns your "standard" into a Frankenstein's Monster of data exchange, and you'd think we'd have learned over the last few decades that ambiguity in specification of a communication protocol is just going to lead to headaches and incompatible/buggy hardware. It's also weird to say SysEx is needed for backwards compat with MIDI 1.0. You can just require a translation layer between MIDI 1.0 sysex and MIDI 2.0 configuration/property exchange for backwards compat. Opaque binary as a part of the protocol does not encourage compatibility among hardware.
    - aikah 7y ago
      The MIDI spec was successful for 30+ years for a reason and manufacturers managed with SysEx without any issue, there is no justification for any "compatibility layer" whatsoever here. The problem wasn't the spec obviously.
      - 7y ago
uhoh-itsmaciek 7y ago
It's been kind of a headache in Firefox's WebMIDI support plans (since SysEx is sometimes used for firmware updates and it's hard to explain "this page may hack your keyboard" in a permission request dialog): https://github.com/mozilla/standards-positions/issues/58#issuecomment-369067271 https://github.com/mozilla/standards-positions/issues/58#iss...

deleted 7y ago

[deleted]

8bitsrule 7y ago

Lots of 1.0 instruments have only SysEx to load/save patches. A single patch may have hundreds of parameters. Whole libraries of patches exist; they can be loaded/saved at once with Sysex. So, no way.

unlinked_dll 7y ago
I don't really see why that matters for a new protocol that fallbacks to MIDI 1.0, it's not a superset of MIDI 1.0. If your device only takes MIDI 1.0 SysEx for patches, anything that would send those patches via MIDI is going to do it on top of the fallback. What I'm saying is a new MIDI 2.0 device probably shouldn't be able to use SysEx. Either opt out of MIDI 1.0 and use the paradigms established or fallback to MIDI 1.0 and its limitations. Otherwise we're just going to get a mess of different implementations, and MIDI 2.0 will fail to be a successful standard.