4 ms·

Hey, I'm the guy who made this! This all started from a random idea I had, but then I was tickled by the thought of becoming faster than the average typist on a

Hey, I'm the guy who made this! This all started from a random idea I had, but then I was tickled by the thought of becoming faster than the average typist on a piano. It all culminated with a livestream today (https://www.twitch.tv/videos/404585365 https://www.twitch.tv/videos/404585365) where I did typing tests for about 3 hours. I'll try to address the comments on here, so ask any questions you have!

Feuilles_Mortes 8y ago
It might be fun/interesting to pipe pieces transcribed to MIDI into your system and see what falls out. I'm sure it'll be nonsense, but perhaps fun nonetheless. (e.g. http://www.piano-midi.de/chopin.htm http://www.piano-midi.de/chopin.htm)
- Adam13531 8y ago
  I made a little explanatory document (https://goo.gl/Vq3Fsb https://goo.gl/Vq3Fsb) to go along with the typing tests that I streamed today, and that was one of the things that I mentioned (copy/pasting from there): Q: Can you play a real song so that we can see what it types out? A: This isn't as satisfying as you may think since the entire typing range is only about two octaves, and most songs exceed that range, so nothing actually gets typed when you play those notes. Anyway, here's Wonderwall: Iuiuigaezobzgbxgebiuiuigaezobzgbegz (that's actually the right hand of Für Elise)
  - Rychard 8y ago
    > Anyway, here's Wonderwall There are many things that I would like to say to you, but I don't know how.
    - coldtea 8y ago
      Write them to him using a piano keyboard
- rhizome 8y ago
  Gah, I tried to get here before someone suggested rendering Moby Dick.
etaioinshrdlu 8y ago
Do not be afraid to release bad source code. It's fine.
kian 8y ago
What were the key or note chording patterns you used to make this?
- Adam13531 8y ago
  I'm not totally sure I understand your question. My original design document (from apparently a year ago!) is here: https://goo.gl/U1HnMu https://goo.gl/U1HnMu ← that was part of what I produced in the "six hours" that I mention in the video. Perhaps that will help, but I don't expect you to read through 11 pages to find your answer, so with clarification, I'd be happy to answer you!
  - kian 8y ago
    Sorry i phrased that poorly - what was the mapping of piano chords to characters you settled on?
    - kian 8y ago
      I went through and read your design document - thanks for the link! That answered my question. For anyone else wondering, one piano key per character within a given modifier palette (like letter, number, etc). With 24 major and minor chords, 4 times that in minor and major sevenths, another 12 dominant chords, 12 augmented, and 12*11 interval pairs, there is a very large space to map interface structures to. The trick is making it harmonically pleasing - however, with a graph of harmonic transitions and cadences, it ought to be possible to find isomorphisms that preserve pleasing sound if you matched unikely letter combinations to unpleasing sounds. Perhaps in the same way that i9en used to infer what you had typed based on probability and scrolling, perhaps we could do something similar with arpeggios or chords played within a certain span.
      - Adam13531 8y ago
        Ah, good, I'm glad you found the answer! As for making something like this harmonically pleasing, it's been touched on here in other threads, but I think the whole project would require a redesign. If you were still aiming for a one-to-one mapping of piano keys to letters, then I think you would end up sacrificing both learning speed and typing speed.
brownbat 8y ago
Love the details for completeness, like modes, and your discussion of adjustments you found necessary along the way. Great job on the project and great explainer video too. Slight tangent--I've often wondered if the principles behind audio keyloggers, where you record typing sounds and try to reconstruct what was typed, could be used to make a wireless computer keyboard that never needs batteries. Seems like you're attacking an inverted version of that problem. Interesting stuff, thanks!
- rijoja 8y ago
  That's a really cool angle. As a matter of fact I'm working on a system like this only that it uses FFT to get the notes instead of midi.
Insanity 8y ago
This was really cool to watch! The youtube video offered a nice explanation and I actually enjoyed the audio/visual-feedback from watching you type on it. Kudos!
rijoja 8y ago
You beat me to it! I've been working on similar thing, so I'm a bit jealous that you published this before me! Then again it makes me happy that I'm not the only one with far out ideas like this! My approach is a tad different though. My approach works by using the microphone. And I have the code to extract notes from a microphone via a FFT. Allowing for typing with say a guitar. Extracting single notes works quite well. I haven't gotten around to test chords just yet, but I think I have the fundamentals there. I'm a bit short on musical knowledge and instruments so so far I've only tested it with my voice (ugh), synthesized sounds and partially filled water bottles. Also I have a suggestion for a graphical interface that would make the learning curve easier and making it language/alphabet independent. Please reply to this comment if you are interested in integrating this.
- jackxbritton 8y ago
  Been doing some pitch detection recently myself, but I went for an average magnitude difference function over an FFT. My understanding is that FFTs are a bit more intensive than autocorrelation-like solutions, but they work better with polytonal signals. Typing seems suited to single-tone instruments - did you consider using AMDF (or a similar function)?
  - rijoja 8y ago
    To be honest all of this is on the border of my knowledge horizon. No I've never heard of AMDF but I will check it out for sure. In the long run as I wrote I am aiming to support chords so that'll be necessery. Would AMDF make the response time quicker? For what purpose are you doing pitch detection?
- Adam13531 8y ago
  Sounds like a fun project! RE: integration with what I did - I'm hoping to release the source code one day, then you'd most likely be free to use that code however you want (I'll probably MIT-license it). But as for my involvement, I'd like to be done with this project.
yomly 8y ago
My friend is a professional pianist and wants to see how high he can get his WPM on typing - is the source available?
- Adam13531 8y ago
  Unfortunately no, not yet, although I'll try to make a post if I ever open-source it!
mindcrash 8y ago
If you love these kind of things you will be blown away by this: https://codeklavier.space/ https://codeklavier.space/ Which can be seen in action here: https://www.youtube.com/watch?v=HrUNrb3ihl0 https://www.youtube.com/watch?v=HrUNrb3ihl0 And no, no April Fools here ;-)

Feuilles_Mortes 8y ago

It might be fun/interesting to pipe pieces transcribed to MIDI into your system and see what falls out. I'm sure it'll be nonsense, but perhaps fun nonetheless. (e.g. http://www.piano-midi.de/chopin.htm http://www.piano-midi.de/chopin.htm)

Adam13531 8y ago
I made a little explanatory document (https://goo.gl/Vq3Fsb https://goo.gl/Vq3Fsb) to go along with the typing tests that I streamed today, and that was one of the things that I mentioned (copy/pasting from there): Q: Can you play a real song so that we can see what it types out? A: This isn't as satisfying as you may think since the entire typing range is only about two octaves, and most songs exceed that range, so nothing actually gets typed when you play those notes. Anyway, here's Wonderwall: Iuiuigaezobzgbxgebiuiuigaezobzgbegz (that's actually the right hand of Für Elise)
- Rychard 8y ago
  > Anyway, here's Wonderwall There are many things that I would like to say to you, but I don't know how.
  - coldtea 8y ago
    Write them to him using a piano keyboard
rhizome 8y ago
Gah, I tried to get here before someone suggested rendering Moby Dick.

etaioinshrdlu 8y ago

Do not be afraid to release bad source code. It's fine.

kian 8y ago

What were the key or note chording patterns you used to make this?

Adam13531 8y ago
I'm not totally sure I understand your question. My original design document (from apparently a year ago!) is here: https://goo.gl/U1HnMu https://goo.gl/U1HnMu ← that was part of what I produced in the "six hours" that I mention in the video. Perhaps that will help, but I don't expect you to read through 11 pages to find your answer, so with clarification, I'd be happy to answer you!
- kian 8y ago
  Sorry i phrased that poorly - what was the mapping of piano chords to characters you settled on?
  - kian 8y ago
    I went through and read your design document - thanks for the link! That answered my question. For anyone else wondering, one piano key per character within a given modifier palette (like letter, number, etc). With 24 major and minor chords, 4 times that in minor and major sevenths, another 12 dominant chords, 12 augmented, and 12*11 interval pairs, there is a very large space to map interface structures to. The trick is making it harmonically pleasing - however, with a graph of harmonic transitions and cadences, it ought to be possible to find isomorphisms that preserve pleasing sound if you matched unikely letter combinations to unpleasing sounds. Perhaps in the same way that i9en used to infer what you had typed based on probability and scrolling, perhaps we could do something similar with arpeggios or chords played within a certain span.
    - Adam13531 8y ago
      Ah, good, I'm glad you found the answer! As for making something like this harmonically pleasing, it's been touched on here in other threads, but I think the whole project would require a redesign. If you were still aiming for a one-to-one mapping of piano keys to letters, then I think you would end up sacrificing both learning speed and typing speed.

brownbat 8y ago

Love the details for completeness, like modes, and your discussion of adjustments you found necessary along the way. Great job on the project and great explainer video too. Slight tangent--I've often wondered if the principles behind audio keyloggers, where you record typing sounds and try to reconstruct what was typed, could be used to make a wireless computer keyboard that never needs batteries. Seems like you're attacking an inverted version of that problem. Interesting stuff, thanks!

rijoja 8y ago
That's a really cool angle. As a matter of fact I'm working on a system like this only that it uses FFT to get the notes instead of midi.

Insanity 8y ago

This was really cool to watch! The youtube video offered a nice explanation and I actually enjoyed the audio/visual-feedback from watching you type on it. Kudos!

rijoja 8y ago

You beat me to it! I've been working on similar thing, so I'm a bit jealous that you published this before me! Then again it makes me happy that I'm not the only one with far out ideas like this! My approach is a tad different though. My approach works by using the microphone. And I have the code to extract notes from a microphone via a FFT. Allowing for typing with say a guitar. Extracting single notes works quite well. I haven't gotten around to test chords just yet, but I think I have the fundamentals there. I'm a bit short on musical knowledge and instruments so so far I've only tested it with my voice (ugh), synthesized sounds and partially filled water bottles. Also I have a suggestion for a graphical interface that would make the learning curve easier and making it language/alphabet independent. Please reply to this comment if you are interested in integrating this.

jackxbritton 8y ago
Been doing some pitch detection recently myself, but I went for an average magnitude difference function over an FFT. My understanding is that FFTs are a bit more intensive than autocorrelation-like solutions, but they work better with polytonal signals. Typing seems suited to single-tone instruments - did you consider using AMDF (or a similar function)?
- rijoja 8y ago
  To be honest all of this is on the border of my knowledge horizon. No I've never heard of AMDF but I will check it out for sure. In the long run as I wrote I am aiming to support chords so that'll be necessery. Would AMDF make the response time quicker? For what purpose are you doing pitch detection?
Adam13531 8y ago
Sounds like a fun project! RE: integration with what I did - I'm hoping to release the source code one day, then you'd most likely be free to use that code however you want (I'll probably MIT-license it). But as for my involvement, I'd like to be done with this project.

yomly 8y ago

My friend is a professional pianist and wants to see how high he can get his WPM on typing - is the source available?

Adam13531 8y ago
Unfortunately no, not yet, although I'll try to make a post if I ever open-source it!

mindcrash 8y ago

If you love these kind of things you will be blown away by this: https://codeklavier.space/ https://codeklavier.space/ Which can be seen in action here: https://www.youtube.com/watch?v=HrUNrb3ihl0 https://www.youtube.com/watch?v=HrUNrb3ihl0 And no, no April Fools here ;-)