7 ms·

Right on, I couldn’t agree more. We are living during a period of exponential progress. I like AlphaCode’s approach of using language models with search. In the

Right on, I couldn’t agree more. We are living during a period of exponential progress. I like AlphaCode’s approach of using language models with search. In the last year I have experimented with mixing language models for NLP with semantic web/linked data tasks, so much simpler than what AlphaCode does, but I have been having fun. I have added examples for this in new additions to two of my books, but if you want a 90 second play-time, here is a simple colab notebook https://colab.research.google.com/drive/1FX-0eizj2vayXsqfSB2ONuJYG8BaYpGO?usp=sharing https://colab.research.google.com/drive/1FX-0eizj2vayXsqfSB2... We live in exciting times. Given exponential rates of progress, I can’t really even imagine what breakthroughs we will see in the next year, let alone the next five years. EDIT: I forgot to mention: having GitHub/OpenAI tools like CoPilot always running in the background in PyCharm and VSCode has in a few short months changed my workflow, for the better.

skybrian 5y ago
Could you say a bit more about how you use CoPilot? What are the "other tools" you use?
- mark_l_watson 5y ago
  I installed the VSCode and PyCharm CoPilot plugins, and signed in with my GitHub account (you first need to request CoPilot access and wait until you get it). As you type in comments or code, CoPilot will sometime autocomplete up to about 10 lines of code, based on the content of the file you are editing (maybe just the code close to the edit point?). My other tools? I use LispWorks Professional, Emacs with Haskell support, and sometimes IntelliJ. I work a lot on remote servers so I depend on SSH/Mosh and tmux also.
  - skybrian 5y ago
    Do you usually find the autocomplete from CoPilot useful enough to keep without editing, or is it wrong often enough to be annoying?
    - zarzavat 5y ago
      For me both are true. The suggestions are usually annoying and wrong, however sometimes copilot makes an observation that is non-obvious, i.e it suggests code that makes me realize that I was about to write a subtle bug. I keep it on as a sanity check.
    - mark_l_watson 5y ago
      If it is wrong, I hit the escape key and the suggested code disappears.
    - crummy 5y ago
      In my opinion it's wrong most of the time but it's so harmless that it's absolutely worth it for the times it's right. Kind of like regular autocomplete. If it's wrong you just keep typing.
stephc_int13 5y ago
We're not seeing exponential rates of progress, in my opinion. We've been on the steep part of an S-curve.
- CuriouslyC 5y ago
  100% this. Exponential progress is only a thing if potential progress is infinite. If the potential progress is finite (hint: it is), eventually the rate of progress hit progressively diminishing returns.
  - globalise83 5y ago
    What is the limit you foresee on computation, especially when such computations can optimise themselves without human intervention?
    - stephc_int13 5y ago
      The underlying rule here, in my opinion, is the law of diminishing returns. (log shaped curve) AlphaZero is already capable of optimizing itself in the limited problem space of Chess. Infinitely increasing the computing power of this system won't give it properties it does not already have, there is no singularity point to be found ahead. And I am not sure that there are any singularities lying ahead in any other domains with the current approach of ML/AI.
      - notahacker 5y ago
        And building on that, the real bottleneck in most domains isn't going to be computer power, it's going to be human understanding of how to curate data or tweak parameters. We've already seen in games with simple rules and win conditions that giving computers data on what we think are good human games can make them perform worse than not giving them data. Most problems aren't possible for humans to ebncapsulate perfectly in a set of rules and win conditions to just leave the processing power to fill in the details, and whilst curating data and calibrating learning processes is an area we've improved hugely on to get where we are with ML, it's not something where human knowledge seems more likely to reach an inflection point than hit diminishing returns.
vain_cain 5y ago
I mostly work with data mining on my personal projects(which is a couple of hours every day), and I'm pretty sure I didn't have to write a single regex since I've started using Copilot. It's hard for me to even imagine how I used to do it before, and how much time I've wasted on stupid mistakes and typos. Now I just write a comment of what I want and an example string. It does the job without me having to modify anything 99% of the time, even for complex stuff. Sure, sometimes it gives an overly complicated solution but it almost always works. Exciting times.
- mark_l_watson 5y ago
  Thanks for the good example. The first time I tried OpenAI's assistant (before I had access to GitHub CoPilot) I wrote a comment in Javascript code that read something like "perform SPARQL query to find out where Bill Gates works" and code was generated using an appropriate library, the SPARQL (like SQL) query was correct, and the code worked. Blew my mind.
- 37ef_ced3 5y ago
  How do you know the regular expressions are correct, without understanding them?
  - incrudible 5y ago
    All regular expressions are incorrect, but some are useful.
    - 37ef_ced3 5y ago
      - alar44 5y ago
    - incrudible 5y ago
      (It's a reference) https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong

skybrian 5y ago

Could you say a bit more about how you use CoPilot? What are the "other tools" you use?

mark_l_watson 5y ago
I installed the VSCode and PyCharm CoPilot plugins, and signed in with my GitHub account (you first need to request CoPilot access and wait until you get it). As you type in comments or code, CoPilot will sometime autocomplete up to about 10 lines of code, based on the content of the file you are editing (maybe just the code close to the edit point?). My other tools? I use LispWorks Professional, Emacs with Haskell support, and sometimes IntelliJ. I work a lot on remote servers so I depend on SSH/Mosh and tmux also.
- skybrian 5y ago
  Do you usually find the autocomplete from CoPilot useful enough to keep without editing, or is it wrong often enough to be annoying?
  - zarzavat 5y ago
    For me both are true. The suggestions are usually annoying and wrong, however sometimes copilot makes an observation that is non-obvious, i.e it suggests code that makes me realize that I was about to write a subtle bug. I keep it on as a sanity check.
  - mark_l_watson 5y ago
    If it is wrong, I hit the escape key and the suggested code disappears.
  - crummy 5y ago
    In my opinion it's wrong most of the time but it's so harmless that it's absolutely worth it for the times it's right. Kind of like regular autocomplete. If it's wrong you just keep typing.

stephc_int13 5y ago

We're not seeing exponential rates of progress, in my opinion. We've been on the steep part of an S-curve.

CuriouslyC 5y ago
100% this. Exponential progress is only a thing if potential progress is infinite. If the potential progress is finite (hint: it is), eventually the rate of progress hit progressively diminishing returns.
- globalise83 5y ago
  What is the limit you foresee on computation, especially when such computations can optimise themselves without human intervention?
  - stephc_int13 5y ago
    The underlying rule here, in my opinion, is the law of diminishing returns. (log shaped curve) AlphaZero is already capable of optimizing itself in the limited problem space of Chess. Infinitely increasing the computing power of this system won't give it properties it does not already have, there is no singularity point to be found ahead. And I am not sure that there are any singularities lying ahead in any other domains with the current approach of ML/AI.
    - notahacker 5y ago
      And building on that, the real bottleneck in most domains isn't going to be computer power, it's going to be human understanding of how to curate data or tweak parameters. We've already seen in games with simple rules and win conditions that giving computers data on what we think are good human games can make them perform worse than not giving them data. Most problems aren't possible for humans to ebncapsulate perfectly in a set of rules and win conditions to just leave the processing power to fill in the details, and whilst curating data and calibrating learning processes is an area we've improved hugely on to get where we are with ML, it's not something where human knowledge seems more likely to reach an inflection point than hit diminishing returns.

vain_cain 5y ago

I mostly work with data mining on my personal projects(which is a couple of hours every day), and I'm pretty sure I didn't have to write a single regex since I've started using Copilot. It's hard for me to even imagine how I used to do it before, and how much time I've wasted on stupid mistakes and typos. Now I just write a comment of what I want and an example string. It does the job without me having to modify anything 99% of the time, even for complex stuff. Sure, sometimes it gives an overly complicated solution but it almost always works. Exciting times.

mark_l_watson 5y ago
Thanks for the good example. The first time I tried OpenAI's assistant (before I had access to GitHub CoPilot) I wrote a comment in Javascript code that read something like "perform SPARQL query to find out where Bill Gates works" and code was generated using an appropriate library, the SPARQL (like SQL) query was correct, and the code worked. Blew my mind.
37ef_ced3 5y ago
How do you know the regular expressions are correct, without understanding them?
- incrudible 5y ago
  All regular expressions are incorrect, but some are useful.
  - 37ef_ced3 5y ago
    - alar44 5y ago
  - incrudible 5y ago
    (It's a reference) https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong