You will be very interested in my colleague Brad Myers' work on Natural Programming. http://www.cs.cmu.edu/~NatProg/ http://www.cs.cmu.edu/~NatProg/ In partic

You will be very interested in my colleague Brad Myers' work on Natural Programming. http://www.cs.cmu.edu/~NatProg/ http://www.cs.cmu.edu/~NatProg/ In particular, one of his PhD students John Pane did a number of studies looking at how people would naturally describe the behaviors in Pac Man. It was really nice work. You should also take a look at Programming by Demonstration. One book, Watch What I Do, is published online in its entirety by the editor Allen Cypher, see here http://acypher.com/wwid/ http://acypher.com/wwid/. Another book in this space, Your Wish is My Command, is also published online https://web.media.mit.edu/~lieber/Your-Wish/ https://web.media.mit.edu/~lieber/Your-Wish/ Both of these are collections of research papers showing how people can demonstrate what they want to a computer, and have the computer figure out what they want. There's a more recent book on programming by demonstration, but I don't recall the title. But definitely check out the first link on Natural Programming, Brad Myers is one of the world experts on this topic.