3 ms·

Ha, I knew I wouldn't be the only person adding ChatGPT to emacs (https://github.com/CarlQLange/chatgpt-arcana.el https://github.com/CarlQLange/chatgpt-arcana.e

Ha, I knew I wouldn't be the only person adding ChatGPT to emacs (https://github.com/CarlQLange/chatgpt-arcana.el https://github.com/CarlQLange/chatgpt-arcana.el). I am beginning to think that ChatGPT is the singularity. I built that emacs package, but really, ChatGPT built it using me as a conduit - there isn't a single function in there that ChatGPT didn't write the majority of (although the poor code quality is all my own). It could go further. Using something like langchain, I'm pretty sure you could get ChatGPT to instruct itself in building a python library or something: User request: build a python library to print out ascii charts in the terminal ChatGPT2ChatGPT: What are the steps I need to take to build a python library to print out charts? For step in steps: ChatGPT2ChatGPT: Carry out this instruction And so forth. I'm sure some human interaction is required, particularly given the context limit. But even so. Another thought I had was that using ChatGPT to help me build things is so convenient that I am unlikely to pick technology that it doesn't know about. Perhaps this kind of thing will cause a chilling effect in the creation of new tech. Who knows. Truly exciting times.

williamcotton 4y ago
While LLMs will move the industry to well defined specs in English that are used to generate code and tests ranging from unit, integration and end-to-end, I don’t see the end of the role of humans in this process. I see the continued rise of the product engineer who is fully capable of interacting with stakeholders and customers while also having the time and energy to produce working software. The singularity seems pretty far off and the current LLM techniques will change drastically over the coming years… Algol families exist but we’ve well beyond those abstractions. One thing I’ve noticed is that LLMs will eventually run in to Syntax errors or other logical errors and halt on their own. I’ve been calling it the non-halting problem and intuitively it seems as though this will be a rather large hurdle to overcome! There’s a certain amount of computational thinking required by a developer required to understand how the computer is interpreting code that seems missing from the capabilities of both LLMs and symbolic interpreters like traditional computers.
- redfloatplane 4y ago
  I think it's likely that the edges of generative AI haven't really been found yet. I think it's hard to say whether any given (textual) requirement definitively can't in some way be fulfilled by ChatGPT (or a similar LLM) without at least another several months of messing around. I certainly haven't made up my mind about what this might do to the world in general - more a question for science fiction authors than programmers, I think. In any case, I am fairly unworried (and more excited) by these developments. ChatGPT can't stop me from sitting on a mountain and eating a banana. Yet.
  - williamcotton 4y ago
    I agree. My goal, which seems very achievable from my current research, is to write formal specifications in English and generate functioning code. One of my favorite recent projects is called Parsel: Parsel : A (De-)compositional Framework for Algorithmic Reasoning with Language Models https://arxiv.org/abs/2212.10561 https://arxiv.org/abs/2212.10561 Here's a notebook with an introduction: https://github.com/ezelikman/parsel/blob/main/parsel.ipynb https://github.com/ezelikman/parsel/blob/main/parsel.ipynb And here's a GUI interface the author has been developing: http://zelikman.me/parsel/interface.html http://zelikman.me/parsel/interface.html
pmoriarty 4y ago
What's needed to close the loop are unit tests written by chatgpt.
- bob1029 4y ago
  I've found that chatgpt is pretty OK at writing basic unit tests based upon the code in context. I can even say things like 'please switch to mstest/xunit' and it does the thing like you'd expect.
- redfloatplane 4y ago
  TabNine actually announced that functionality a few weeks ago for their code-completion tool: https://www.tabnine.com/blog/ai-unit-test-generation/ https://www.tabnine.com/blog/ai-unit-test-generation/ I haven't used it, but it might be what you're looking for. I'm actually pretty sure ChatGPT can write effective tests as well, though.

williamcotton 4y ago

While LLMs will move the industry to well defined specs in English that are used to generate code and tests ranging from unit, integration and end-to-end, I don’t see the end of the role of humans in this process. I see the continued rise of the product engineer who is fully capable of interacting with stakeholders and customers while also having the time and energy to produce working software. The singularity seems pretty far off and the current LLM techniques will change drastically over the coming years… Algol families exist but we’ve well beyond those abstractions. One thing I’ve noticed is that LLMs will eventually run in to Syntax errors or other logical errors and halt on their own. I’ve been calling it the non-halting problem and intuitively it seems as though this will be a rather large hurdle to overcome! There’s a certain amount of computational thinking required by a developer required to understand how the computer is interpreting code that seems missing from the capabilities of both LLMs and symbolic interpreters like traditional computers.

redfloatplane 4y ago
I think it's likely that the edges of generative AI haven't really been found yet. I think it's hard to say whether any given (textual) requirement definitively can't in some way be fulfilled by ChatGPT (or a similar LLM) without at least another several months of messing around. I certainly haven't made up my mind about what this might do to the world in general - more a question for science fiction authors than programmers, I think. In any case, I am fairly unworried (and more excited) by these developments. ChatGPT can't stop me from sitting on a mountain and eating a banana. Yet.
- williamcotton 4y ago
  I agree. My goal, which seems very achievable from my current research, is to write formal specifications in English and generate functioning code. One of my favorite recent projects is called Parsel: Parsel : A (De-)compositional Framework for Algorithmic Reasoning with Language Models https://arxiv.org/abs/2212.10561 https://arxiv.org/abs/2212.10561 Here's a notebook with an introduction: https://github.com/ezelikman/parsel/blob/main/parsel.ipynb https://github.com/ezelikman/parsel/blob/main/parsel.ipynb And here's a GUI interface the author has been developing: http://zelikman.me/parsel/interface.html http://zelikman.me/parsel/interface.html

pmoriarty 4y ago

What's needed to close the loop are unit tests written by chatgpt.

bob1029 4y ago
I've found that chatgpt is pretty OK at writing basic unit tests based upon the code in context. I can even say things like 'please switch to mstest/xunit' and it does the thing like you'd expect.
redfloatplane 4y ago
TabNine actually announced that functionality a few weeks ago for their code-completion tool: https://www.tabnine.com/blog/ai-unit-test-generation/ https://www.tabnine.com/blog/ai-unit-test-generation/ I haven't used it, but it might be what you're looking for. I'm actually pretty sure ChatGPT can write effective tests as well, though.