4 ms·

Piano playing transformed into Playstation controllers that play StreetFighter

176 points by crousto 12y ago

tchap 12y ago
Nice !!
deleted 12y ago
[deleted]
deleted 12y ago
[deleted]
scott_s 12y ago
The title, and even their own abstract, does not do this project justice. They did not simply map piano keys to a controller's buttons. About a quarter through the post: A simple solution would be to link every piano key to a controler button. This could be fairly easy to build, but seemed that it would lean way too much towards non-music, providing too much of a simple translation, instead of a metaphor. We wanted a system that would allow us to say: “If you’re a good pianist, then you’ll be a good gamer”. ... Simply put, we had to find a way to translate what a good pianist would play into what a good gamer would play. This talk then continues much later: To tackle this situation, we decided to adopt a clear and strict separation of concerns, in which our firmware would be conceptual center of command: * the single role of the piano is to play notes (key hits) * the single role of the console is to play controls (button hits) * the role of the pianette, our firmware, is to receive notes from the piano, somehow turn them into controls, and send these controls to the console That became the basis for a refactor of the whole firmware. A one-to-one mapping from piano keys to controller buttons would be interesting, but not... satisfying. They really thought about this deeply, and mapped music to fighting.
- dang 12y ago
  We changed the submission title to one that attempts to capture more of this magnificence. If anyone suggests a better, we'll change it again.
- xom 12y ago
  They tried, but I'm not convinced that they succeeded. Their video resembles Street Fighter in the way that https://youtu.be/-2gJamguN04 https://youtu.be/-2gJamguN04 resembles Football. It's possible that their mapping works, in that there's some non-unpleasant piano sequence that maps to at least superficially convincing gameplay, but their video didn't show it. The connection of analog (piano) to digital is impressive. The additional claim of having mapped music to fighting is not yet substantiated.
  - Retra 12y ago
    They may not have succeeded, but they did build an infrastructure that could probably handle a true success.
  - hurin 12y ago
    It's neat that they can play a duet that results in actual gameplay; but generally I think you are correct, a player trying only to win would inevitably play something pretty awful sounding. (It would be hard to establish a musical parallel to game play on one piano - let alone two!)
    - smtucker 12y ago
      Someone could do this in reverse, right? Take the demo of a fight between skilled players and derive what keys they would have had to press on the piano to make it happen. I know that some (most modern fighters?) fighting games have options to show inputs on screen.
      - hurin 12y ago
        Sortof. For one if it could be done it would only work for that particular fight. Even that is highly unlikely though. For one consider that music at least in terms how we (culturally, or biologically) are predisposed to listening to it: must keep time, you can change from 4/4 to 5/4 in the next section, but not arbitrarily (if you want it to sound like what people are inclined to call music at least). (i.e. the drum-pad as a controller video someone linked in one of the responses, which is totally non-musical - is a good example). Of course in a live-fighting game you don't care about structured timing, only about relative timing as to what your opponent is doing. Perhaps you could run a game through an emulator that enforced actions into particular time-blocks. Anyways I think it's very difficult, even to get something satisfactory rhythmically, not to mention tonality and a second piano.
- ryanjmo 12y ago
  I totally get what you are saying and I'm glad for the effort they put in, but... I'd almost rather see a 1-1 mapping of controllers to piano keys and see people hacking on the piano, to see how good Street Fighter play corresponds to sound. Even if it sounded terrible, I think that would be more interesting to me than this produced video, which I don't feel like I pulled a lot of factual information out of.
- crousto 12y ago
  (project and article co-author here) Thank you scott_s for quoting _these_ words from the post. They are our most sincere attempt to express why we even put so much effort developing the idea. It is great and humbling that they come across as such.
- jpwagner 12y ago
  Reminds me of some of the work of Xenakis: http://en.wikipedia.org/wiki/Game_piece_(music) http://en.wikipedia.org/wiki/Game_piece_(music)
zrail 12y ago
I'm so very glad I read the whole article before watching the video. The journey to get to that duel is incredible.
- foobarge 12y ago
  Yes - also: "The concept was pitched for the reopening of the Maison de la Radio, French’s principal and historical radio building situated in the heart of Paris." \o/ - everybody loves Maison de la Radio, home of the excellent French radio public broadcasting (France Inter, France Musique and especially France Culture - which is some serious public broadcasting with no compromise on quality.)
ForHackernews 12y ago
This guy beat Dark Souls with a drum controller: https://www.youtube.com/watch?v=02my_zhX4Bs https://www.youtube.com/watch?v=02my_zhX4Bs
bonn1 12y ago
Is there no video of the guys playing Street Fighter on the piano?
wellec 12y ago
Happy birthday OP
Scrub1 12y ago
Awesome work! Reminded me of this: https://www.threadless.com/product/1773/ https://www.threadless.com/product/1773/
Phithagoras 12y ago
It would have been even better if they had been able to rig it so that the commands to operate the game roughly followed the rules of classical harmony, that way, the gameplay would be more obviously musical. As it is, it sounds a bit atonal.

176 points by crousto 12y ago

tchap 12y ago

Nice !!

deleted 12y ago

[deleted]

deleted 12y ago

[deleted]

scott_s 12y ago

The title, and even their own abstract, does not do this project justice. They did not simply map piano keys to a controller's buttons. About a quarter through the post: A simple solution would be to link every piano key to a controler button. This could be fairly easy to build, but seemed that it would lean way too much towards non-music, providing too much of a simple translation, instead of a metaphor. We wanted a system that would allow us to say: “If you’re a good pianist, then you’ll be a good gamer”. ... Simply put, we had to find a way to translate what a good pianist would play into what a good gamer would play. This talk then continues much later: To tackle this situation, we decided to adopt a clear and strict separation of concerns, in which our firmware would be conceptual center of command: * the single role of the piano is to play notes (key hits) * the single role of the console is to play controls (button hits) * the role of the pianette, our firmware, is to receive notes from the piano, somehow turn them into controls, and send these controls to the console That became the basis for a refactor of the whole firmware. A one-to-one mapping from piano keys to controller buttons would be interesting, but not... satisfying. They really thought about this deeply, and mapped music to fighting.

dang 12y ago
We changed the submission title to one that attempts to capture more of this magnificence. If anyone suggests a better, we'll change it again.
xom 12y ago
They tried, but I'm not convinced that they succeeded. Their video resembles Street Fighter in the way that https://youtu.be/-2gJamguN04 https://youtu.be/-2gJamguN04 resembles Football. It's possible that their mapping works, in that there's some non-unpleasant piano sequence that maps to at least superficially convincing gameplay, but their video didn't show it. The connection of analog (piano) to digital is impressive. The additional claim of having mapped music to fighting is not yet substantiated.
- Retra 12y ago
  They may not have succeeded, but they did build an infrastructure that could probably handle a true success.
- hurin 12y ago
  It's neat that they can play a duet that results in actual gameplay; but generally I think you are correct, a player trying only to win would inevitably play something pretty awful sounding. (It would be hard to establish a musical parallel to game play on one piano - let alone two!)
  - smtucker 12y ago
    Someone could do this in reverse, right? Take the demo of a fight between skilled players and derive what keys they would have had to press on the piano to make it happen. I know that some (most modern fighters?) fighting games have options to show inputs on screen.
    - hurin 12y ago
      Sortof. For one if it could be done it would only work for that particular fight. Even that is highly unlikely though. For one consider that music at least in terms how we (culturally, or biologically) are predisposed to listening to it: must keep time, you can change from 4/4 to 5/4 in the next section, but not arbitrarily (if you want it to sound like what people are inclined to call music at least). (i.e. the drum-pad as a controller video someone linked in one of the responses, which is totally non-musical - is a good example). Of course in a live-fighting game you don't care about structured timing, only about relative timing as to what your opponent is doing. Perhaps you could run a game through an emulator that enforced actions into particular time-blocks. Anyways I think it's very difficult, even to get something satisfactory rhythmically, not to mention tonality and a second piano.
ryanjmo 12y ago
I totally get what you are saying and I'm glad for the effort they put in, but... I'd almost rather see a 1-1 mapping of controllers to piano keys and see people hacking on the piano, to see how good Street Fighter play corresponds to sound. Even if it sounded terrible, I think that would be more interesting to me than this produced video, which I don't feel like I pulled a lot of factual information out of.
crousto 12y ago
(project and article co-author here) Thank you scott_s for quoting _these_ words from the post. They are our most sincere attempt to express why we even put so much effort developing the idea. It is great and humbling that they come across as such.
jpwagner 12y ago
Reminds me of some of the work of Xenakis: http://en.wikipedia.org/wiki/Game_piece_(music) http://en.wikipedia.org/wiki/Game_piece_(music)

zrail 12y ago

I'm so very glad I read the whole article before watching the video. The journey to get to that duel is incredible.

foobarge 12y ago
Yes - also: "The concept was pitched for the reopening of the Maison de la Radio, French’s principal and historical radio building situated in the heart of Paris." \o/ - everybody loves Maison de la Radio, home of the excellent French radio public broadcasting (France Inter, France Musique and especially France Culture - which is some serious public broadcasting with no compromise on quality.)

ForHackernews 12y ago

This guy beat Dark Souls with a drum controller: https://www.youtube.com/watch?v=02my_zhX4Bs https://www.youtube.com/watch?v=02my_zhX4Bs

bonn1 12y ago

Is there no video of the guys playing Street Fighter on the piano?

wellec 12y ago

Happy birthday OP

Scrub1 12y ago

Awesome work! Reminded me of this: https://www.threadless.com/product/1773/ https://www.threadless.com/product/1773/

Phithagoras 12y ago

It would have been even better if they had been able to rig it so that the commands to operate the game roughly followed the rules of classical harmony, that way, the gameplay would be more obviously musical. As it is, it sounds a bit atonal.