4 ms·
I can’t help you with actually converting speech to code but it occurs to me this would be a benefit to everyone. Speaking the words that are represented by th
by mulmen 6y ago
I can’t help you with actually converting speech to code but it occurs to me this would be a benefit to everyone. Speaking the words that are represented by the code we write would require a much deeper understanding of what we are doing and why.
Food for thought for sure. Good luck.
- lunixbochs 6y agoThere's been some pretty cool work in this area recently: https://www.youtube.com/watch?v=fZSFNUT6iY8 https://www.youtube.com/watch?v=fZSFNUT6iY8
- Udik 6y agoThat's seriously cool. I think that a tool that could take a high level description of a small piece of code and write it for me (it doesn't have to be correct- I can double check it and fix it quicker than I write it) would make my coding much quicker and less stressful. There are so many times in which I think: "I know exactly how to do this, step by step: I'm just annoyed at having to type it down for the umpteenth time".
- mulmen 6y agoWow. This is exactly what I imagined but it already exists. This is a great illustration of how we as humans could use our abilities to disambiguate to collaborate with a computer and write code. Very impressive stuff. To me it seems like learning how to talk to Alexa or Cortana or "Google" is a limitation or regression for humans. This shows that it could actually be beneficial. Thanks for this philosophical rabbit hole just in time for a weekend.
- lunixbochs 6y agoAs someone who works in voice tech, I think talking to Alexa is setting back our expectations of voice tech by at least a decade. The actual tech and capabilities we have available right now are so much better than static capabilities over a high latency internet connection.
- raidicy 6y agosorry i am using dictation software. I have found this research paper[0] that describes using a machine translation with a CNN and a r n n. it seems one of the main problems that come up with is a lack of training data so to progress this further it seems like we would need a big corpus of parallel text between natural English describing what the code does in the actual source code. I only have some novice experience with training machine learning models through transfer learning with fast AI but I have found a repository of tutorials[1] that teach you how to do machine translation between two languages and it seems maybe this approach might be applicable. There is a small corpus of text[2] with python code with annotations but and the paper itself at States. It probably needs more to get a higher than a 74% accuracy. the only thing I can really think is to scrape GitHub make a website where you can get people to crowdsource annotations for code samples. I'm not sure on how to deal with how much and how little code would be for a sampling. I eat how much of the code would you need for a good training sample versus just using isolated functions. Perhaps that something like this might be used in correspondence with GPT. [0]https://www.groundai.com/project/machine-translation-from-natural-language-to-code-using-long-short-term-memory/1 https://www.groundai.com/project/machine-translation-from-na... [1]https://github.com/bentrevett/pytorch-seq2seq https://github.com/bentrevett/pytorch-seq2seq [2]https://ahclab.naist.jp/pseudogen/ https://ahclab.naist.jp/pseudogen/