6 ms·

I am using copilots now since a few months and it really makes me a 2x more productive developer. Its like you become an orchestrator of a dev team. You still n

I am using copilots now since a few months and it really makes me a 2x more productive developer. Its like you become an orchestrator of a dev team. You still need to look into details, but things just flow much much faster. I can only imagine how it will be like if I also have access to a AI debugger / end to end tester. Completing the loop and making it super efficient. Also think that this is not only the case for developers. Even for lawyer it could be the same thing. Expected production output of a worker is going to rise. The ones who do not embrace AI assistants in the near future will have a hard time in the future.

zdragnar 3y ago
> Even for lawyer it could be the same thing Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt. It's useful for rote things, but for anything that you depend on, you still need to give it just as much attention as if you'd done it yourself.
- qup 3y ago
  I agree, but it does change the shape of the work you would be doing. Validating work isn't the same as generating new work. Whether that's better, or useful, probably differs by situation.
  - blibble 3y ago
    > Validating work isn't the same as generating new work. in many cases it's way harder
- cloverich 3y ago
  > you still need to give it just as much attention as if you'd done it yourself. It's different than a Lawyers case, where the facts require manually cross referencing. In our case, that verification can come via directly and immediately running the code. How good the actual code is varies, but the fundamental difference remains.
- skissane 3y ago
  > Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt. For something like legal work, you don’t want a raw LLM. You want an agent integrated with a legal database, in such a way that it can’t cite cases which don’t exist in the database. And it can’t generate a direct quote from a case unless that text actually occurs in the case. You still need a human lawyer familiar with the law to pick up on subtler errors, but better technology can prevent the grosser ones. And the subtler errors (e.g. misrepresenting what a case says through partial or out of context quoting) is the kind of error human lawyers sometimes make too - like every other profession, lawyers vary greatly in their competence, and often only the grosser cases of incompetence incur sanctions
  - zmgsabst 3y ago
    Tried Lexis+ AI, and… it’s just not very good yet. Much like ChatGPT, it can only handle recall and summation — and even then, I don’t fully trust it because it often misses key ideas. And much like ChatGPT, it can’t do anything coherent at length without a lot of help and working around its faults. And seems entirely unaware of similar sounding words having distinct legal meanings. Which is not good.
    - skissane 3y ago
      > Tried Lexis+ AI, and… it’s just not very good yet. I wonder to what extent that’s due to current inherent limitations of the technology, and to what extent it is due to quality of implementation issues. It is hard to say because (I presume) there is only limited public information on how it is actually implemented. e.g. which LLM is it using? How much fine-tuning has been done? Are they using other potentially helpful techniques such as guided sampling? Or breaking down the task into parts and having multiple agents each specialised to handle one particular part? > Much like ChatGPT When you compare it to ChatGPT, do you mean GPT3.5 or GPT4? I also would guess that certain areas of law (especially criminal law) may be prone to triggering “safeguards” which result in poorer performance than if those safeguards were absent. Arguing that what your client did was legal (even be it ethically unsavoury) is an essential part of a lawyer’s job
unshavedyak 3y ago
I still haven't figured out how people use this that much. I use LLMs almost daily via ChatGPT, Phind, Kagi Ultimate (my current one), etc. However i spend too much time pushing the LLM towards my goal that i can't imagine it speeding my coding up. I clearly find value in LLMs to some degree, but speeding up my coding is not yet one.. i'd love to, but i just don't understand. I can only imagine typing more in explanation for the LLM than it would take me to write it to begin with.
- oblio 3y ago
  IMO a chunk of it is like Peel developers: people that prioritize their own speed above all else: readability, team reviews, etc. I'd love to be proven wrong.
- 010101010101 3y ago
  I rarely find interfacing with an external chat interface useful, but integration with the coding environment (e.g. Copilot) is an immediate productivity boost.
- swatcoder 3y ago
  Some revered novelists spend all day to write a single page and others are able to get in a flow and produce a chapter in that same time. Without getting into touchy questions of quality and talent and the whole 10x trope, there are a lot of working engineers out there that produce code a lot more slowly than others and that work on more common problems than others. My sense is copilot-like products provide the biggest boon to those people and are a lot harder for people who are more naturally prolific or who work on more esoteric things.
  - holoduke 3y ago
    Have you ever tried it?. It sounds like you are a bit against the idea of an AI supporting your work.
- freedomben 3y ago
  It all seems to depend on what you are doing. The more niche and technical the task, the less the AI can help. If you are just generating crud and points for a common web language and framework, it can be an 80% boost. I think the real problem with this is that people aren't differentiating these different types of work when they give these numbers.
rglover 3y ago
> The ones who do not embrace AI assistants in the near future will have a hard time in the future. The exact opposite will be true and the funny (sad?) part is that they will lack the skills necessary (because they got lazy and over-trusted the AI) to fix mistakes/incompatible solutions.
- fhd2 3y ago
  My thinking as well. If coding assistants become so good that I'm at a competitive disadvantage, I'll just start using them. It's not rocket science. So far, everything I've tried largely slowed me down. As a Google/SO replacement for some types of questions, they sure save me maybe an hour per week, but that's really all I could extract so far. Maybe my work is not too typical though, I spend only a fraction of my time actually typing in code. And I do eliminate the need for boilerplate through other means (picking frameworks/libraries that are a good fit for the problem, refactoring, meta programming, scripts, suitable tool chain etc).
  - zmgsabst 3y ago
    1 hour per week of increased coding is a 5-7% boost in productivity, using the Amazon guidelines for how SDEs use their time — 50%/20hrs for SDE1 and 33%/13hrs for SDE3. Is that enough to be competitive? I’m not sure — but at scale that would be a 5% reduction in headcount for the same work, or ~$12M/yr for every 1,000 engineers. If you can figure out how to get 2-3 hours more coding done a week, we’re talking real gains.
    - fhd2 3y ago
      Depends on where the time is saved. If it's in figuring out how to do something in Django where StackOverflow is flooded with outdated answers, sure. I see those kinds of savings. But the tragic beauty of programming is that a little time saved today can very well mean lots of time lost later. The former you can measure, the latter is a tougher nut.
- skwirl 3y ago
  People said the same thing about garbage collectors.
taude 3y ago
in the future, it seems like we might just become PR reviewers
rco8786 3y ago
I’ve had it enabled for months across both Javascript and Kotlin codebases and it’s…fine? Good enough that I leave it enabled. But only barely. I’m certainly not orchestrating a dev team. It has probably the same productivity boost that intellisense gave back when it came out. Which is good, but still marginal. Certainly not replacing anyone’s job.

zdragnar 3y ago

> Even for lawyer it could be the same thing Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt. It's useful for rote things, but for anything that you depend on, you still need to give it just as much attention as if you'd done it yourself.

qup 3y ago
I agree, but it does change the shape of the work you would be doing. Validating work isn't the same as generating new work. Whether that's better, or useful, probably differs by situation.
- blibble 3y ago
  > Validating work isn't the same as generating new work. in many cases it's way harder
cloverich 3y ago
> you still need to give it just as much attention as if you'd done it yourself. It's different than a Lawyers case, where the facts require manually cross referencing. In our case, that verification can come via directly and immediately running the code. How good the actual code is varies, but the fundamental difference remains.
skissane 3y ago
> Funny you mention that, since lawyers have already gotten in trouble for citing fictional cases when submitting work performed by chatgpt. For something like legal work, you don’t want a raw LLM. You want an agent integrated with a legal database, in such a way that it can’t cite cases which don’t exist in the database. And it can’t generate a direct quote from a case unless that text actually occurs in the case. You still need a human lawyer familiar with the law to pick up on subtler errors, but better technology can prevent the grosser ones. And the subtler errors (e.g. misrepresenting what a case says through partial or out of context quoting) is the kind of error human lawyers sometimes make too - like every other profession, lawyers vary greatly in their competence, and often only the grosser cases of incompetence incur sanctions
- zmgsabst 3y ago
  Tried Lexis+ AI, and… it’s just not very good yet. Much like ChatGPT, it can only handle recall and summation — and even then, I don’t fully trust it because it often misses key ideas. And much like ChatGPT, it can’t do anything coherent at length without a lot of help and working around its faults. And seems entirely unaware of similar sounding words having distinct legal meanings. Which is not good.
  - skissane 3y ago
    > Tried Lexis+ AI, and… it’s just not very good yet. I wonder to what extent that’s due to current inherent limitations of the technology, and to what extent it is due to quality of implementation issues. It is hard to say because (I presume) there is only limited public information on how it is actually implemented. e.g. which LLM is it using? How much fine-tuning has been done? Are they using other potentially helpful techniques such as guided sampling? Or breaking down the task into parts and having multiple agents each specialised to handle one particular part? > Much like ChatGPT When you compare it to ChatGPT, do you mean GPT3.5 or GPT4? I also would guess that certain areas of law (especially criminal law) may be prone to triggering “safeguards” which result in poorer performance than if those safeguards were absent. Arguing that what your client did was legal (even be it ethically unsavoury) is an essential part of a lawyer’s job

unshavedyak 3y ago

I still haven't figured out how people use this that much. I use LLMs almost daily via ChatGPT, Phind, Kagi Ultimate (my current one), etc. However i spend too much time pushing the LLM towards my goal that i can't imagine it speeding my coding up. I clearly find value in LLMs to some degree, but speeding up my coding is not yet one.. i'd love to, but i just don't understand. I can only imagine typing more in explanation for the LLM than it would take me to write it to begin with.

oblio 3y ago
IMO a chunk of it is like Peel developers: people that prioritize their own speed above all else: readability, team reviews, etc. I'd love to be proven wrong.
010101010101 3y ago
I rarely find interfacing with an external chat interface useful, but integration with the coding environment (e.g. Copilot) is an immediate productivity boost.
swatcoder 3y ago
Some revered novelists spend all day to write a single page and others are able to get in a flow and produce a chapter in that same time. Without getting into touchy questions of quality and talent and the whole 10x trope, there are a lot of working engineers out there that produce code a lot more slowly than others and that work on more common problems than others. My sense is copilot-like products provide the biggest boon to those people and are a lot harder for people who are more naturally prolific or who work on more esoteric things.
- holoduke 3y ago
  Have you ever tried it?. It sounds like you are a bit against the idea of an AI supporting your work.
freedomben 3y ago
It all seems to depend on what you are doing. The more niche and technical the task, the less the AI can help. If you are just generating crud and points for a common web language and framework, it can be an 80% boost. I think the real problem with this is that people aren't differentiating these different types of work when they give these numbers.

rglover 3y ago

> The ones who do not embrace AI assistants in the near future will have a hard time in the future. The exact opposite will be true and the funny (sad?) part is that they will lack the skills necessary (because they got lazy and over-trusted the AI) to fix mistakes/incompatible solutions.

fhd2 3y ago
My thinking as well. If coding assistants become so good that I'm at a competitive disadvantage, I'll just start using them. It's not rocket science. So far, everything I've tried largely slowed me down. As a Google/SO replacement for some types of questions, they sure save me maybe an hour per week, but that's really all I could extract so far. Maybe my work is not too typical though, I spend only a fraction of my time actually typing in code. And I do eliminate the need for boilerplate through other means (picking frameworks/libraries that are a good fit for the problem, refactoring, meta programming, scripts, suitable tool chain etc).
- zmgsabst 3y ago
  1 hour per week of increased coding is a 5-7% boost in productivity, using the Amazon guidelines for how SDEs use their time — 50%/20hrs for SDE1 and 33%/13hrs for SDE3. Is that enough to be competitive? I’m not sure — but at scale that would be a 5% reduction in headcount for the same work, or ~$12M/yr for every 1,000 engineers. If you can figure out how to get 2-3 hours more coding done a week, we’re talking real gains.
  - fhd2 3y ago
    Depends on where the time is saved. If it's in figuring out how to do something in Django where StackOverflow is flooded with outdated answers, sure. I see those kinds of savings. But the tragic beauty of programming is that a little time saved today can very well mean lots of time lost later. The former you can measure, the latter is a tougher nut.
skwirl 3y ago
People said the same thing about garbage collectors.

taude 3y ago

in the future, it seems like we might just become PR reviewers

rco8786 3y ago

I’ve had it enabled for months across both Javascript and Kotlin codebases and it’s…fine? Good enough that I leave it enabled. But only barely. I’m certainly not orchestrating a dev team. It has probably the same productivity boost that intellisense gave back when it came out. Which is good, but still marginal. Certainly not replacing anyone’s job.