3 ms·

I think this is what's happening. Code production speed has increased. But the speed and efficiency of review and testing hasn't kept pace. Also, my current jo

I think this is what's happening. Code production speed has increased. But the speed and efficiency of review and testing hasn't kept pace. Also, my current job limits the use of AI coding tools, because of the risk of our IP leaking out. To really be useful, there is a lot of context a programmer should be able to collate. Not all of it is going to be in the repo.

dathinab 3y ago
IMHO there is some form of crisis of "value production speed" for various programming things. And AI gives you fast code production speed so it seems to be a fix. But due to less structured code and code reuse it needs much more code for the same value produced. And it loves to produce the kind of bugs which are easy to pass reviews and tests (because this are the kind of bugs it most likely has as "correct" code in it's training data). Which means less time per-value for human code reviews and in turn lower quality results on many levels. I originally hoped that with CS/SoftwareDev getting older and less explosive grows and Universities more realizing the (sometimes huge) gap between CS and SoftwareDev the industry would get a chance of consolidating tech and dev processes to fix that issue. But with the rise of LLMs this hope might be further away then it ever had been in the last 20 year.
- stcredzero 3y ago
  Why not devote LLMs to enhancing testing and review?
  - dathinab 3y ago
    the current code generating LLMs seem not suited for this just having more test doesn't mean you actually test more (or more correctly) Having AI specialized on assisting testing is viable but in the end would be done quite different as far as I can tell even if it uses LLMs. Through maybe domain adoption and a very different scaffolding might be enough.
    - stcredzero 3y ago
      They seem suited to doing things like, coming up with a basic test, then modifying it to cover boundary conditions, provided those aren't too abstruse.

dathinab 3y ago

IMHO there is some form of crisis of "value production speed" for various programming things. And AI gives you fast code production speed so it seems to be a fix. But due to less structured code and code reuse it needs much more code for the same value produced. And it loves to produce the kind of bugs which are easy to pass reviews and tests (because this are the kind of bugs it most likely has as "correct" code in it's training data). Which means less time per-value for human code reviews and in turn lower quality results on many levels. I originally hoped that with CS/SoftwareDev getting older and less explosive grows and Universities more realizing the (sometimes huge) gap between CS and SoftwareDev the industry would get a chance of consolidating tech and dev processes to fix that issue. But with the rise of LLMs this hope might be further away then it ever had been in the last 20 year.

stcredzero 3y ago
Why not devote LLMs to enhancing testing and review?
- dathinab 3y ago
  the current code generating LLMs seem not suited for this just having more test doesn't mean you actually test more (or more correctly) Having AI specialized on assisting testing is viable but in the end would be done quite different as far as I can tell even if it uses LLMs. Through maybe domain adoption and a very different scaffolding might be enough.
  - stcredzero 3y ago
    They seem suited to doing things like, coming up with a basic test, then modifying it to cover boundary conditions, provided those aren't too abstruse.