3 ms·

Lines of code are meaningful when taken in aggregate and useless as a metric for an individual’s contributions. COCOMO, which considers lines of code, is gener

Lines of code are meaningful when taken in aggregate and useless as a metric for an individual’s contributions. COCOMO, which considers lines of code, is generally accepted as being accurate (enough) at estimating the value of a software system, at least as far as how courts (in the US) are concerned. https://en.wikipedia.org/wiki/COCOMO https://en.wikipedia.org/wiki/COCOMO

jonahx 7mo ago
> Lines of code are meaningful when taken in aggregate The linked article does not demonstrate this. It establishes no causal link. One can obviously bloat LOC to an arbitrary degree while maintaining feature parity. Very generously, assuming good faith participants, it might reflect a kind average human efficiency within the fixed environment of the time. Carrying the conclusions of this study from the 80s into the LLM age is not justified scientifically.
keeda 7mo ago
> Lines of code are meaningful when taken in aggregate and useless as a metric for an individual’s contributions. Yes, and in fact a lot of the studies that show the impact of AI on coding productivity get dismissed because they use LoC or PRs as a metric and "everyone knows LoC/PR counts is a BS metric." But the better designed of these studies specifically call this out and explicitly design their experiments to use these as aggregate metrics.
sarchertech 7mo ago
No one has any idea how to estimate software value, so the idea that some courts in the US have used a wildly inaccurate system that considers LOC is so far away from evidence that LOC is useful for anything that I can’t believe you bothered including that. LOC is essentially only useful to give a ballpark estimate it complexity and even then only if you compare orders of magnitude and only between similar program languages and ecosystems. It’s certainly not useful for AI generated projects. Just look at OpenClaw. Last I heard it was something close to half a million lines of code. When I was in college we had a professor senior year who was obsessed with COCOMO. He required our final group project to be 50k LOC (He also required that we print out every line and turn it in). We made it, but only because we build a generator for the UI and made sure the generator was as verbose as possible.
- brabel 7mo ago
  They gave a widely accepted way to estimate value, and your counter argument is that that is inaccurate. Fine but how can you be confident about that? I see only one way which is for you to come up with a better way and then show that by your better estimation, COCOMO is bad. Until you do that, all your argument goes down to is vibes. Your example about OpenClaw works exactly against your own argument by the way: OpenAI acquired it for millions by all accounts.
  - sarchertech 7mo ago
    COCOMO has been shown to be inaccurate numerous times. Google it. Here’s one result. “A very high MMRE (1.00) indicates that, on average, the COCOMO model misses about 100% of the actual project effort. This means that the estimate generated by the model can be double or even greater than the actual effort. This shows that the COCOMO model is not able to provide estimates that are close to the actual value.” No one in the industry has taken COCOMO seriously for nearly 2 decades. >OpenClaw 1. OpenAI bought the vibes and the creator. Why would they buy the code? It’s open source. 2. You don’t seriously think OpenClaw needs half a million lines of code to provide the functionality it does do you? Seriously just go look at the code. No one is defending that as being an efficient use of code. https://journal.fkpt.org/index.php/BIT/article/download/2027/907 https://journal.fkpt.org/index.php/BIT/article/download/2027...
    - brabel 7mo ago
      > No one in the industry has taken COCOMO seriously for nearly 2 decades. The funny thing is that we've just discussed how people do take it seriously. It's just that you don't like that. And what do you offer as an alternative? Like I said, vibes. You think that the value of some software is something you can only "feel". That's not how an engineer thinks. If you're engineer you should know that if you can't measure it, you can't say anything at all about it. Which means you cannot discount any alternative method until you've got a better way. But clearly you can't think like an engineer.
      - 7mo ago
BoorishBears 7mo ago
> at least as far as how courts (in the US) are concerned. That's an anti-signal if we're being honest.
post-it 7mo ago
I think that's a "looking under the lamp post because that's where the light is" metric. I'm not sure most developers, managers, or owners care about the calculated dollar value of their codebase. They're not trading code on an exchange. By condensing all software into a scalar, you're losing almost all important information. I can see why it's important in court, obviously, since civil court is built around condensing everything into a scalar.
renegade-otter 7mo ago
I am writing a book! I used AI to write 1 billion words this morning!
kqr 7mo ago
COCOMO estimates the cost of the software, not the value. The cost is only weakly correlated with value.
coldtea 7mo ago
>at least as far as how courts are concerned. Courts would be the last place to understand something like code quality or software project value....

jonahx 7mo ago

> Lines of code are meaningful when taken in aggregate The linked article does not demonstrate this. It establishes no causal link. One can obviously bloat LOC to an arbitrary degree while maintaining feature parity. Very generously, assuming good faith participants, it might reflect a kind average human efficiency within the fixed environment of the time. Carrying the conclusions of this study from the 80s into the LLM age is not justified scientifically.

keeda 7mo ago

> Lines of code are meaningful when taken in aggregate and useless as a metric for an individual’s contributions. Yes, and in fact a lot of the studies that show the impact of AI on coding productivity get dismissed because they use LoC or PRs as a metric and "everyone knows LoC/PR counts is a BS metric." But the better designed of these studies specifically call this out and explicitly design their experiments to use these as aggregate metrics.

sarchertech 7mo ago

No one has any idea how to estimate software value, so the idea that some courts in the US have used a wildly inaccurate system that considers LOC is so far away from evidence that LOC is useful for anything that I can’t believe you bothered including that. LOC is essentially only useful to give a ballpark estimate it complexity and even then only if you compare orders of magnitude and only between similar program languages and ecosystems. It’s certainly not useful for AI generated projects. Just look at OpenClaw. Last I heard it was something close to half a million lines of code. When I was in college we had a professor senior year who was obsessed with COCOMO. He required our final group project to be 50k LOC (He also required that we print out every line and turn it in). We made it, but only because we build a generator for the UI and made sure the generator was as verbose as possible.

brabel 7mo ago
They gave a widely accepted way to estimate value, and your counter argument is that that is inaccurate. Fine but how can you be confident about that? I see only one way which is for you to come up with a better way and then show that by your better estimation, COCOMO is bad. Until you do that, all your argument goes down to is vibes. Your example about OpenClaw works exactly against your own argument by the way: OpenAI acquired it for millions by all accounts.
- sarchertech 7mo ago
  COCOMO has been shown to be inaccurate numerous times. Google it. Here’s one result. “A very high MMRE (1.00) indicates that, on average, the COCOMO model misses about 100% of the actual project effort. This means that the estimate generated by the model can be double or even greater than the actual effort. This shows that the COCOMO model is not able to provide estimates that are close to the actual value.” No one in the industry has taken COCOMO seriously for nearly 2 decades. >OpenClaw 1. OpenAI bought the vibes and the creator. Why would they buy the code? It’s open source. 2. You don’t seriously think OpenClaw needs half a million lines of code to provide the functionality it does do you? Seriously just go look at the code. No one is defending that as being an efficient use of code. https://journal.fkpt.org/index.php/BIT/article/download/2027/907 https://journal.fkpt.org/index.php/BIT/article/download/2027...
  - brabel 7mo ago
    > No one in the industry has taken COCOMO seriously for nearly 2 decades. The funny thing is that we've just discussed how people do take it seriously. It's just that you don't like that. And what do you offer as an alternative? Like I said, vibes. You think that the value of some software is something you can only "feel". That's not how an engineer thinks. If you're engineer you should know that if you can't measure it, you can't say anything at all about it. Which means you cannot discount any alternative method until you've got a better way. But clearly you can't think like an engineer.
    - 7mo ago

BoorishBears 7mo ago

> at least as far as how courts (in the US) are concerned. That's an anti-signal if we're being honest.

post-it 7mo ago

I think that's a "looking under the lamp post because that's where the light is" metric. I'm not sure most developers, managers, or owners care about the calculated dollar value of their codebase. They're not trading code on an exchange. By condensing all software into a scalar, you're losing almost all important information. I can see why it's important in court, obviously, since civil court is built around condensing everything into a scalar.

renegade-otter 7mo ago

I am writing a book! I used AI to write 1 billion words this morning!

kqr 7mo ago

COCOMO estimates the cost of the software, not the value. The cost is only weakly correlated with value.

coldtea 7mo ago

>at least as far as how courts are concerned. Courts would be the last place to understand something like code quality or software project value....