9 ms·

Build or seek out a MIDI generating model. I hope Stable Audio is never the place for that. MIDI is deeply lossy and it would be tragedy if it was the only mu

Build or seek out a MIDI generating model. I hope Stable Audio is never the place for that. MIDI is deeply lossy and it would be tragedy if it was the only music representation. Imagine if instead of phonographs, compact disks and streaming audio we only had piano rolls. What a loss indeed.

iainctduncan 3y ago
Midi is not lossy, midi is symbolic. There's a huge difference.
- Applejinx 3y ago
  No, it's lossy. It's an event model at a fixed data rate. You can only do so many things sequentially, even if you could represent any possible musical concept as a MIDI event. So even if you're not sticking to note-on, note-off, it's still extremely lossy.
  - gamblor956 3y ago
    MIDI is able to accommodate nearly everything that can be represented through a musical score and instrumental performance. What are you hoping to accomplish with AI-generated waveforms that can't be done with MIDI?
    - waffletower 3y ago
      The problems have long been known and articulated: http://www.music.mcgill.ca/~gary/courses/papers/Moore-Dysfunctions-CMJ-1988.pdf http://www.music.mcgill.ca/~gary/courses/papers/Moore-Dysfun...
      - gamblor956 3y ago
        Your example of the failures of MIDI are based on a 35-year old paper (from 1988!!!) about an earlier version of MIDI? When that paper was written it took several weeks and many millions of dollars of equipment to render primitive, mono-color 3d graphics. Desktop computers had 512 kilobytes of RAM and the highest-end desktops 32 MB of hard drive storage space. Computer screens had two colors: black and green. Audio cards capable of making beeps and clicks were the cutting-edge. WIFI was still a decade away.
        waffletower 3y ago
        You clearly didn't read the article, and clearly don't understand how prevalent the MIDI 1.0 specification is today. MIDI 2.0 is a very recent development (this year LOL!) and has yet to be commercially adopted. The 1984 design is what is largely in use today. At the time of initial development, commercial synthesizers, not sound cards, were the intended generators of sound utilizing MIDI: https://www.vintagesynth.com/roland/juno106.php https://www.vintagesynth.com/roland/juno106.php.
        PaulDavisThe1st 3y ago
        MPE, which is the model for the most important part of MIDI 2.0, has been around for several years now; there are numerous software synths (and a few hardware ones) that can use it, and several hardware controllers ("instruments") that can deliver it.
    - mdp2021 3y ago
      > What "Intention" (as a tentative term) The question becomes: what has impeded the creation of a MIDI file that can be confused with an actual concert from Arturo Benedetti Michelangeli.
      - gamblor956 3y ago
        Literally nothing is preventing this other than that nobody has bothered to take the time to do it. The current version of MIDI is capable of replicating any of his performances, even down to the randomness. Note that if you want to replicate the audio quality of his performances, you will need a high-quality MIDI instrument; the ones that ship with Windows will not suffice. These MIDI instruments can range from a few dollars to thousands of dollars. (See, e.g., Native Instruments)
        mdp2021 3y ago
        > nobody has bothered to take the time to do it In such case, we have a theoretical suggestion that «nothing is preventing this», but not an actual proof based on a "Turing test"-like scenario which would have specialists fooled, to corroborate that the new MIDI 2 would suffice.
        gamblor956 3y ago
        I gave you the knowledge to do this research yourself, but since you are unwilling to do so, here is an example of the performance possible using MIDI instruments: https://www.youtube.com/watch?v=CvaChiq6gf0 https://www.youtube.com/watch?v=CvaChiq6gf0 As it is clear that you intend to keep shifting the goalposts to make a point that can't be made, I will withdraw from further participation in this thread.
    - waffletower 3y ago
      This is absurd. Sure, someone below posits that MIDI could perhaps represent a piano performance by Arturo Benedetti Michelangeli. I think it has been able to do a passing job at that, when you provide a decent piano. Regardless, piano rolls have been able to come close since the early 20th century. But how well does MIDI represent music performed by John Coltrane? Jimi Hendrix? It falls on its face. The long fetishized Western music notation abstraction, which MIDI poorly simulates, completely fails for many important examples of music. I would even venture to say that MIDI fails for most of them. But yes, MIDI is well-optimized for piano music where an acceptable piano or simulation is available.
    - toasternz 3y ago
      These diffusion models give an ability to create far more than can be communicated with MIDI + instrument. Riffusion gave a hint of this - rather than just notes and drum hits + some processing, it becomes one big pulsating, expressive mass which would not be reproducible without the granularity of a diffusion model. These are reminiscent of some of the serendipity of live recordings with lots of tracks where interesting things happen from interplay of many different layers. Generating a few dozen clips generally would give me 2 or 3 with a beautiful emotional passage which really lights up the pleasurable music part of my brain. Mass farming these clips seems like a good route to some amazing music.
    - Applejinx 3y ago
      Absolutely not. I can write a score where 16 people each strike a xylophone note at the same instant. I pick xylophone because it's a very sharp transient, and in theory the performers could perform so well that it's literally one attack. If that's not enough, make a XM file with 16 channels and guarantee that they'll all be the same attack, or run a modular synth and some mults and trigger it off the rise time of a clock pulse, no computer involved. Midi has to fire all the notes one at a time. You cannot represent even everything represented by a score through MIDI because it's serial and sends one message at a time. Stretch that out to 16 sharp clicky attacks and you'll notice that MIDI cannot fire them all at once, it'll blur.
      - gamblor956 3y ago
        The MIDI1 spec allows for up to 16 channels on each port, and would definitely be capable of handling 16 xylophones at once. It's complicated, but the current MIDI spec (and the pre-spec MPE extension) combines data, so what you think of as 16 xylophones is to the MIDI device 1 instrument consisting of 16 xylophones. However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem like a real instrument and not a computer-generated tone, then you would need to dedicate some of the 16 channels on each port to each of those different controls. To handle 16 xylophones, you would need a MIDI interface with enough ports to handle the note data plus however many control channels you want to use. (Note that from the interface's perspective, the data incoming from 1 port is just a single channel, so an "8-channel" MIDI interface is actually handling 128 channels of MIDI data.)
        Applejinx 3y ago
        You're not following me. It's a single serial bus at 3,125 bytes per second. Expressive controls like polyphonic pressure take at least two bytes, sometimes three: that's for a single message. A note on is typically three bytes, channel, key and velocity. All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a chain of voltage pulses synchronize individual sequencers and aren't themselves notes. 1000 messages a second (at three bytes for each note-on) seems like a lot but it really isn't. With your 16 instruments (any drum, the xylophones, whatever) you can fire about 62 notes on all instruments per second. That seems like a lot too, but it's a hard limit, and it means your sixteen instruments have to 'blur' across 16 milliseconds to all fire off a note. That means every time you fire all the instruments as one click, instead the attacks make up a 960hz tone. That's NOT one attack. Humans on percussion instruments can do better than that. It's the equivalent of 5.5 feet of space between speakers playing back: if you're time-aligning a tweeter and a midrange to produce a unified click, and you misalign one of the drivers by putting it five and a half feet back from where it would be, you'll notice the misalignment. Midi's timing issues are also noticeable.
  - miohtama 3y ago
    MIDI 2.0 improves a lot of things, how much dynamicity and variation you can have. MIDI 1.0 is a standard from early 80s. It indeed has shortcomings, but also the upside is editability. It’s then for the remaster / musician / actual interpretation / post production to make the score to something less of an event model.
  - iainctduncan 3y ago
    well ok, you could say it's a lossy format for capturing physical movement, which it certainly is. My point is that it is not a format for capturing music any more than a score on a page is a format for capturing music. Both are instructions for a performer of music (one machine, one human), which is a very different beast.
- Jeff_Brown 3y ago
  MIDI is not inherently lossy. You could encode anything in it, just as you can encode any novel as an integer. In practice, though, transformations from audio to MIDI discard an enormous amount of important information, with the possible exception of transcriptions of performances on piano (where volume, frequency, duration and a good physical model of a piano are enough to reconstruct everything important about the signal) and similar instruments.
  - PaulDavisThe1st 3y ago
    MPE/MIDI 2.0 pretty much theoretically fixes the "discard an enormous amount of important information", in that you can have an extremely large number of parameters describing every single note and/or physical performance nuance. Contemporary MIDI is ready to describe almost any human performance in an almost absurd level of detail. Whether anything can do something useful with the description is a different story.
- waffletower 3y ago
  MIDI is an extraordinarily lossy music representation. Even Claude Shannon would facepalm at the assertion that it could, in theory, represent audio faithfully. It is not its purpose, it is decidedly not its practice, and it is a ludicrously irrelevant example of pedantry to say otherwise. The false equivalency asserted by the commons can be aggravating :D
  - iainctduncan 3y ago
    MIDI is not a lossy format for audio because it's not a representation of audio, period. It's a format for conveying the motion of a piano keyboard, meant from the beginning to be usable for various forms of audio.

iainctduncan 3y ago

Midi is not lossy, midi is symbolic. There's a huge difference.

Applejinx 3y ago
No, it's lossy. It's an event model at a fixed data rate. You can only do so many things sequentially, even if you could represent any possible musical concept as a MIDI event. So even if you're not sticking to note-on, note-off, it's still extremely lossy.
- gamblor956 3y ago
  MIDI is able to accommodate nearly everything that can be represented through a musical score and instrumental performance. What are you hoping to accomplish with AI-generated waveforms that can't be done with MIDI?
  - waffletower 3y ago
    The problems have long been known and articulated: http://www.music.mcgill.ca/~gary/courses/papers/Moore-Dysfunctions-CMJ-1988.pdf http://www.music.mcgill.ca/~gary/courses/papers/Moore-Dysfun...
    - gamblor956 3y ago
      Your example of the failures of MIDI are based on a 35-year old paper (from 1988!!!) about an earlier version of MIDI? When that paper was written it took several weeks and many millions of dollars of equipment to render primitive, mono-color 3d graphics. Desktop computers had 512 kilobytes of RAM and the highest-end desktops 32 MB of hard drive storage space. Computer screens had two colors: black and green. Audio cards capable of making beeps and clicks were the cutting-edge. WIFI was still a decade away.
      - waffletower 3y ago
        You clearly didn't read the article, and clearly don't understand how prevalent the MIDI 1.0 specification is today. MIDI 2.0 is a very recent development (this year LOL!) and has yet to be commercially adopted. The 1984 design is what is largely in use today. At the time of initial development, commercial synthesizers, not sound cards, were the intended generators of sound utilizing MIDI: https://www.vintagesynth.com/roland/juno106.php https://www.vintagesynth.com/roland/juno106.php.
        PaulDavisThe1st 3y ago
        MPE, which is the model for the most important part of MIDI 2.0, has been around for several years now; there are numerous software synths (and a few hardware ones) that can use it, and several hardware controllers ("instruments") that can deliver it.
  - mdp2021 3y ago
    > What "Intention" (as a tentative term) The question becomes: what has impeded the creation of a MIDI file that can be confused with an actual concert from Arturo Benedetti Michelangeli.
    - gamblor956 3y ago
      Literally nothing is preventing this other than that nobody has bothered to take the time to do it. The current version of MIDI is capable of replicating any of his performances, even down to the randomness. Note that if you want to replicate the audio quality of his performances, you will need a high-quality MIDI instrument; the ones that ship with Windows will not suffice. These MIDI instruments can range from a few dollars to thousands of dollars. (See, e.g., Native Instruments)
      - mdp2021 3y ago
        > nobody has bothered to take the time to do it In such case, we have a theoretical suggestion that «nothing is preventing this», but not an actual proof based on a "Turing test"-like scenario which would have specialists fooled, to corroborate that the new MIDI 2 would suffice.
        gamblor956 3y ago
        I gave you the knowledge to do this research yourself, but since you are unwilling to do so, here is an example of the performance possible using MIDI instruments: https://www.youtube.com/watch?v=CvaChiq6gf0 https://www.youtube.com/watch?v=CvaChiq6gf0 As it is clear that you intend to keep shifting the goalposts to make a point that can't be made, I will withdraw from further participation in this thread.
  - waffletower 3y ago
    This is absurd. Sure, someone below posits that MIDI could perhaps represent a piano performance by Arturo Benedetti Michelangeli. I think it has been able to do a passing job at that, when you provide a decent piano. Regardless, piano rolls have been able to come close since the early 20th century. But how well does MIDI represent music performed by John Coltrane? Jimi Hendrix? It falls on its face. The long fetishized Western music notation abstraction, which MIDI poorly simulates, completely fails for many important examples of music. I would even venture to say that MIDI fails for most of them. But yes, MIDI is well-optimized for piano music where an acceptable piano or simulation is available.
  - toasternz 3y ago
    These diffusion models give an ability to create far more than can be communicated with MIDI + instrument. Riffusion gave a hint of this - rather than just notes and drum hits + some processing, it becomes one big pulsating, expressive mass which would not be reproducible without the granularity of a diffusion model. These are reminiscent of some of the serendipity of live recordings with lots of tracks where interesting things happen from interplay of many different layers. Generating a few dozen clips generally would give me 2 or 3 with a beautiful emotional passage which really lights up the pleasurable music part of my brain. Mass farming these clips seems like a good route to some amazing music.
  - Applejinx 3y ago
    Absolutely not. I can write a score where 16 people each strike a xylophone note at the same instant. I pick xylophone because it's a very sharp transient, and in theory the performers could perform so well that it's literally one attack. If that's not enough, make a XM file with 16 channels and guarantee that they'll all be the same attack, or run a modular synth and some mults and trigger it off the rise time of a clock pulse, no computer involved. Midi has to fire all the notes one at a time. You cannot represent even everything represented by a score through MIDI because it's serial and sends one message at a time. Stretch that out to 16 sharp clicky attacks and you'll notice that MIDI cannot fire them all at once, it'll blur.
    - gamblor956 3y ago
      The MIDI1 spec allows for up to 16 channels on each port, and would definitely be capable of handling 16 xylophones at once. It's complicated, but the current MIDI spec (and the pre-spec MPE extension) combines data, so what you think of as 16 xylophones is to the MIDI device 1 instrument consisting of 16 xylophones. However, if you want to control attack, reverb, tone, volume, i.e., all the stuff that makes it seem like a real instrument and not a computer-generated tone, then you would need to dedicate some of the 16 channels on each port to each of those different controls. To handle 16 xylophones, you would need a MIDI interface with enough ports to handle the note data plus however many control channels you want to use. (Note that from the interface's perspective, the data incoming from 1 port is just a single channel, so an "8-channel" MIDI interface is actually handling 128 channels of MIDI data.)
      - Applejinx 3y ago
        You're not following me. It's a single serial bus at 3,125 bytes per second. Expressive controls like polyphonic pressure take at least two bytes, sometimes three: that's for a single message. A note on is typically three bytes, channel, key and velocity. All these messages have to take turns. It's not a tracker, or a modular synth, where you can parallel click rise times electronically: it's not DINsync, where a chain of voltage pulses synchronize individual sequencers and aren't themselves notes. 1000 messages a second (at three bytes for each note-on) seems like a lot but it really isn't. With your 16 instruments (any drum, the xylophones, whatever) you can fire about 62 notes on all instruments per second. That seems like a lot too, but it's a hard limit, and it means your sixteen instruments have to 'blur' across 16 milliseconds to all fire off a note. That means every time you fire all the instruments as one click, instead the attacks make up a 960hz tone. That's NOT one attack. Humans on percussion instruments can do better than that. It's the equivalent of 5.5 feet of space between speakers playing back: if you're time-aligning a tweeter and a midrange to produce a unified click, and you misalign one of the drivers by putting it five and a half feet back from where it would be, you'll notice the misalignment. Midi's timing issues are also noticeable.
- miohtama 3y ago
  MIDI 2.0 improves a lot of things, how much dynamicity and variation you can have. MIDI 1.0 is a standard from early 80s. It indeed has shortcomings, but also the upside is editability. It’s then for the remaster / musician / actual interpretation / post production to make the score to something less of an event model.
- iainctduncan 3y ago
  well ok, you could say it's a lossy format for capturing physical movement, which it certainly is. My point is that it is not a format for capturing music any more than a score on a page is a format for capturing music. Both are instructions for a performer of music (one machine, one human), which is a very different beast.
Jeff_Brown 3y ago
MIDI is not inherently lossy. You could encode anything in it, just as you can encode any novel as an integer. In practice, though, transformations from audio to MIDI discard an enormous amount of important information, with the possible exception of transcriptions of performances on piano (where volume, frequency, duration and a good physical model of a piano are enough to reconstruct everything important about the signal) and similar instruments.
- PaulDavisThe1st 3y ago
  MPE/MIDI 2.0 pretty much theoretically fixes the "discard an enormous amount of important information", in that you can have an extremely large number of parameters describing every single note and/or physical performance nuance. Contemporary MIDI is ready to describe almost any human performance in an almost absurd level of detail. Whether anything can do something useful with the description is a different story.
waffletower 3y ago
MIDI is an extraordinarily lossy music representation. Even Claude Shannon would facepalm at the assertion that it could, in theory, represent audio faithfully. It is not its purpose, it is decidedly not its practice, and it is a ludicrously irrelevant example of pedantry to say otherwise. The false equivalency asserted by the commons can be aggravating :D
- iainctduncan 3y ago
  MIDI is not a lossy format for audio because it's not a representation of audio, period. It's a format for conveying the motion of a piano keyboard, meant from the beginning to be usable for various forms of audio.