5 ms·

Program Synthesis Demo

126 points by Houshalter 12y ago

opless 12y ago
And a link to his homepage where you might find more work on program synthesis http://research.microsoft.com/en-us/um/people/sumitg/ http://research.microsoft.com/en-us/um/people/sumitg/
krilnon 12y ago
The demo is neat, although the flashy, impressive part (NL->program) predates this particular piece of work. For example, Greg Little's "sloppy programming" is from 2008-ish: http://groups.csail.mit.edu/uid/projects/keyword-commands/index.html http://groups.csail.mit.edu/uid/projects/keyword-commands/in... Le and Gulwani's paper does cite Little, but the refutation is a bit weak. > Little and Miller propose a code completion tool that synthesizes the most likely Java expression in a code context from a set of keywords [23]. Smart-Synth is different, in that it synthesizes a complete script and does not require extra contextual information. From the video, a "complete script" appears to be 1 or 2 simple expressions. And not requiring contextual information sounds like it can't take advantage of contextual information.
covi 12y ago
I have attended one of Sumit's guest lectures, and his work on Flash Fill is pretty cool.
robgibbons 12y ago
Very interesting demo. It reminds me of Microsoft Australia's work with on(X)... but this is closer to code completion on steroids.
isbadawi 12y ago
Pretty neat. I noticed something a bit off about the "tint a random photo blue" example around 2:22. Two of the three search results are fine, but the second search result is pics := media -> pictures pic := pics -> random pic -> tint (colors -> sepia) c := colors -> blue All the components are there but not in the right order -- it still tints with the default color.
- Leszek 12y ago
  Isn't that the point though? That the search results show different possible interpretations of the text entry, and this is one of the possible ones?
  - wutbrodo 12y ago
    I wouldn't say "Tint a random photo sepia, and then set an arbitrary unrelated variable to blue" is a possible interpretation of the text "Tint a random photo blue".
itamarhaber 12y ago
Am I the only one hearing Daft Punk's https://www.youtube.com/watch?v=XfH3erWacsQ https://www.youtube.com/watch?v=XfH3erWacsQ?
frik 12y ago
This system could be used together with speech recognition (Siri, Cortana, etc. personal agent) to automate tasks and write little apps on a mobile device.
agumonkey 12y ago
The animated (translation) transitions seems everything but ergonomic to me. For instance, static touch controls grid are pretty efficient, see: http://www.malacria.fr/pdf/gutwin14-fasttap.pdf http://www.malacria.fr/pdf/gutwin14-fasttap.pdf http://www.malacria.fr/video/fasttap-video.mp4 http://www.malacria.fr/video/fasttap-video.mp4
- Uncompetative 12y ago
  Interesting research. Of course, you could always hack the Home button to act as a Command key and hold the device in landscape orientation without having to give up a piece of the screen.
chton 12y ago
I expect this will lead to a whole new category of bugs: "I wasn't precise enough in saying what I wanted, so now the code doesn't do what it should." Interesting technology, but I doubt it will ever replace or even be used in a proper piece of code.
- webwarrior 12y ago
  But it would be a good alternative to bash one-liners.
  - chton 12y ago
    oh absolutely, if they can get it general enough to do that. If they manage to include this in powershell (since it's MS), they could have the worlds greatest command line. "Get all current running processes, select the ones that use over 5% CPU, and display them to me sorted descending by CPU usage" More verbose than the normal, but damn it would make things easier for the non-initiated.
    - mrec 12y ago
      The danger is that you wind up with another AppleScript - something that might read more easily for the non-initiated, but is no easier to write, because it only accepts one particular finicky subset of natural language while dishonestly implying that it'll magically understand all of it.
      - chton 12y ago
        I would hope that Microsoft is smarter and has learned from those mistakes. It does seem to be their intention to do it with full natural language processing. We'll have to wait and see.
- tree_of_item 12y ago
  You put in a query and get out some code. Concrete code not connected to the query anymore. The code doesn't shift out from under you later, if that's what you're implying. Why wouldn't it be used in a "proper piece of code"? You ask Visual Studio/XCode/Eclipse for something and it gives you back a snippet that you can use as a starting point.
beemoe 12y ago
So this is what we were all working on at ICFP last year :)

126 points by Houshalter 12y ago

And a link to his homepage where you might find more work on program synthesis http://research.microsoft.com/en-us/um/people/sumitg/ http://research.microsoft.com/en-us/um/people/sumitg/

krilnon 12y ago

The demo is neat, although the flashy, impressive part (NL->program) predates this particular piece of work. For example, Greg Little's "sloppy programming" is from 2008-ish: http://groups.csail.mit.edu/uid/projects/keyword-commands/index.html http://groups.csail.mit.edu/uid/projects/keyword-commands/in... Le and Gulwani's paper does cite Little, but the refutation is a bit weak. > Little and Miller propose a code completion tool that synthesizes the most likely Java expression in a code context from a set of keywords [23]. Smart-Synth is different, in that it synthesizes a complete script and does not require extra contextual information. From the video, a "complete script" appears to be 1 or 2 simple expressions. And not requiring contextual information sounds like it can't take advantage of contextual information.

covi 12y ago

I have attended one of Sumit's guest lectures, and his work on Flash Fill is pretty cool.

robgibbons 12y ago

Very interesting demo. It reminds me of Microsoft Australia's work with on(X)... but this is closer to code completion on steroids.

isbadawi 12y ago

Pretty neat. I noticed something a bit off about the "tint a random photo blue" example around 2:22. Two of the three search results are fine, but the second search result is pics := media -> pictures pic := pics -> random pic -> tint (colors -> sepia) c := colors -> blue All the components are there but not in the right order -- it still tints with the default color.

Leszek 12y ago
Isn't that the point though? That the search results show different possible interpretations of the text entry, and this is one of the possible ones?
- wutbrodo 12y ago
  I wouldn't say "Tint a random photo sepia, and then set an arbitrary unrelated variable to blue" is a possible interpretation of the text "Tint a random photo blue".

itamarhaber 12y ago

Am I the only one hearing Daft Punk's https://www.youtube.com/watch?v=XfH3erWacsQ https://www.youtube.com/watch?v=XfH3erWacsQ?

frik 12y ago

This system could be used together with speech recognition (Siri, Cortana, etc. personal agent) to automate tasks and write little apps on a mobile device.

agumonkey 12y ago

The animated (translation) transitions seems everything but ergonomic to me. For instance, static touch controls grid are pretty efficient, see: http://www.malacria.fr/pdf/gutwin14-fasttap.pdf http://www.malacria.fr/pdf/gutwin14-fasttap.pdf http://www.malacria.fr/video/fasttap-video.mp4 http://www.malacria.fr/video/fasttap-video.mp4

Uncompetative 12y ago
Interesting research. Of course, you could always hack the Home button to act as a Command key and hold the device in landscape orientation without having to give up a piece of the screen.

chton 12y ago

I expect this will lead to a whole new category of bugs: "I wasn't precise enough in saying what I wanted, so now the code doesn't do what it should." Interesting technology, but I doubt it will ever replace or even be used in a proper piece of code.

webwarrior 12y ago
But it would be a good alternative to bash one-liners.
- chton 12y ago
  oh absolutely, if they can get it general enough to do that. If they manage to include this in powershell (since it's MS), they could have the worlds greatest command line. "Get all current running processes, select the ones that use over 5% CPU, and display them to me sorted descending by CPU usage" More verbose than the normal, but damn it would make things easier for the non-initiated.
  - mrec 12y ago
    The danger is that you wind up with another AppleScript - something that might read more easily for the non-initiated, but is no easier to write, because it only accepts one particular finicky subset of natural language while dishonestly implying that it'll magically understand all of it.
    - chton 12y ago
      I would hope that Microsoft is smarter and has learned from those mistakes. It does seem to be their intention to do it with full natural language processing. We'll have to wait and see.
tree_of_item 12y ago
You put in a query and get out some code. Concrete code not connected to the query anymore. The code doesn't shift out from under you later, if that's what you're implying. Why wouldn't it be used in a "proper piece of code"? You ask Visual Studio/XCode/Eclipse for something and it gives you back a snippet that you can use as a starting point.

beemoe 12y ago

So this is what we were all working on at ICFP last year :)