4 ms·

I can imagine a future where new software is, mainly, written by humans while legacy software is maintained by ai. It can take months for a new developer to un

I can imagine a future where new software is, mainly, written by humans while legacy software is maintained by ai. It can take months for a new developer to understand a legacy code base but an LLM with a big enough context window would be instantly productive.

Yahivin 3y ago
I can imagine the opposite.
- encoderer 3y ago
  Cool tell me more.
  - deleted 3y ago
    [deleted]
  - SteveNuts 3y ago
    Not OP but I could see some shops pushing AI generated code to production, then when changes need to be made, they can't get the AI to modify the existing code in just the way they need, so a human has to intervene.
  - bastawhiz 3y ago
    I can't get Copilot to generate Python that adds numbers together correctly sometimes. Getting an LLM to generate correct, working code for a language that hardly anybody writes anymore is almost assuredly going to lead to failure.
    - encoderer 3y ago
      yeah I agree but when you look at the slope not the y-intercept it’s getting obviously better. one advantage the government would have is training/fine-tuning on a hundred million lines of domain specific cobol.
      - bastawhiz 3y ago
        The slope doesn't really matter, because the target is "better than a human, and able to identify and fix its own errors". The slope will decrease as you approach this threshold. It's also wildly bad to plan to train and fine tune on code that you know has bugs. Already we have Copilot generating code with trivial vulnerabilities because that's what it's trained on.
        encoderer 3y ago
        Roughly ~all code has bugs. We were all trained on trivial examples and buggy code - just like llms. Honestly I think editing code will be easier than creating wholly new applications to precise spec.
nradov 3y ago
The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that. Once you have the detailed specifications the coding is relatively easy. Some of that can be partially automated using AI CASE tools, but that only gives a marginal improvement to overall project productivity.
- dragonwriter 3y ago
  > The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that. One of my first one afternoon (for prompt refinement and trying it out) toy GPT-3.5 “applications”—really just a prompt—was having it interview for reauirements and draft and progressively refine a requirements doc in a specified (by reference to an author in popular Agile literature, not by specific templates, so just relying on the model’s general training) format. Its pretty much what convinced me of the broad utility of the technology. I absolutely think that with a bit of effort, GPL-4-128K, a custom RAG framework and/or the Assistants API, you could build something that handles a lot of this. I’m not really bullish near term on LLMs as full replacements, but I can see it as a big force multiplier.
- charlie0 3y ago
  Ding, ding, ding, we have a winner. This. I see issues at my small startup with not having clarity with certain billing items. This issue is not technical, it's the unknown requirements of why things were written in the way they current exist. I can't imagine what kind of bizatine rules they have in the Pentagon that were placed there decades ago, and no one can answer why. Codebase must be littered with Chesterton fences.
- DougBTX 3y ago
  > The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. So lots of tedious repetitive manipulation of language to extract facts and transform them into a systematic format? My prediction: LLMs are going to eat that for breakfast. (Honestly it is a great business idea for someone with the right contacts. Lots of big customers with deep pockets and large private datasets to train against that might now have a solution to a problem which was previously intractable in scale.)

Yahivin 3y ago

I can imagine the opposite.

encoderer 3y ago
Cool tell me more.
- deleted 3y ago
  [deleted]
- SteveNuts 3y ago
  Not OP but I could see some shops pushing AI generated code to production, then when changes need to be made, they can't get the AI to modify the existing code in just the way they need, so a human has to intervene.
- bastawhiz 3y ago
  I can't get Copilot to generate Python that adds numbers together correctly sometimes. Getting an LLM to generate correct, working code for a language that hardly anybody writes anymore is almost assuredly going to lead to failure.
  - encoderer 3y ago
    yeah I agree but when you look at the slope not the y-intercept it’s getting obviously better. one advantage the government would have is training/fine-tuning on a hundred million lines of domain specific cobol.
    - bastawhiz 3y ago
      The slope doesn't really matter, because the target is "better than a human, and able to identify and fix its own errors". The slope will decrease as you approach this threshold. It's also wildly bad to plan to train and fine tune on code that you know has bugs. Already we have Copilot generating code with trivial vulnerabilities because that's what it's trained on.
      - encoderer 3y ago
        Roughly ~all code has bugs. We were all trained on trivial examples and buggy code - just like llms. Honestly I think editing code will be easier than creating wholly new applications to precise spec.

nradov 3y ago

The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that. Once you have the detailed specifications the coding is relatively easy. Some of that can be partially automated using AI CASE tools, but that only gives a marginal improvement to overall project productivity.

dragonwriter 3y ago
> The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that. One of my first one afternoon (for prompt refinement and trying it out) toy GPT-3.5 “applications”—really just a prompt—was having it interview for reauirements and draft and progressively refine a requirements doc in a specified (by reference to an author in popular Agile literature, not by specific templates, so just relying on the model’s general training) format. Its pretty much what convinced me of the broad utility of the technology. I absolutely think that with a bit of effort, GPL-4-128K, a custom RAG framework and/or the Assistants API, you could build something that handles a lot of this. I’m not really bullish near term on LLMs as full replacements, but I can see it as a big force multiplier.
charlie0 3y ago
Ding, ding, ding, we have a winner. This. I see issues at my small startup with not having clarity with certain billing items. This issue is not technical, it's the unknown requirements of why things were written in the way they current exist. I can't imagine what kind of bizatine rules they have in the Pentagon that were placed there decades ago, and no one can answer why. Codebase must be littered with Chesterton fences.
DougBTX 3y ago
> The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. So lots of tedious repetitive manipulation of language to extract facts and transform them into a systematic format? My prediction: LLMs are going to eat that for breakfast. (Honestly it is a great business idea for someone with the right contacts. Lots of big customers with deep pockets and large private datasets to train against that might now have a solution to a problem which was previously intractable in scale.)