3 ms·
How would you present a piano roll to a transformer (e.g. what would be a sample of the sequence)? You could try using a tuple of pitch integers for each time s
by p1esk 7y ago
How would you present a piano roll to a transformer (e.g. what would be a sample of the sequence)? You could try using a tuple of pitch integers for each time step. I'm not sure how big a "vocabulary" would need to be to capture most of the chords (note combinations) - it might actually be comparable in size to a language vocabulary (tens of thousands of words). You could use two channels to capture note onset/offset info (like it was done in a biaxial RNN paper). Or the encoding used for Musenet (with explicit timing info), but somehow I like the idea of "chords as words" better.
- gwern 7y agoYou would simply have a vector of 128 for each timestep (I'm not sure how discrete MIDI goes), and any of the 128 instruments turned on would be non-zero. It's not a dense encoding, by a long shot, but it's quite literal and explicit so you don't have to worry indirectness making it hard to learn (assuming you have a model which can handle such long-range inputs to begin with). Personally, I'd rather try redoing BPE encoding for ABC-MIDI specifically.