5 ms·

When you have built your working product try this prompt: - Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that s

When you have built your working product try this prompt: - Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. Then cry as the ai reveals that it didn't actually do anything close to what it said it did. I call this my million dollar prompt, as in it teaches you just how much you are being fooled.

KronisLV 2mo ago
> Then cry as the ai reveals that it didn't actually do anything close to what it said it did. If using AI to generate code, you told it generate some code, so it did. No amount of "You are an expert developer" or "Make no mistakes" will change the fact that it just generates tokens and has a limited thinking budget. Adversarial review loops of N parallel agents looking at whatever characteristics you care about will make it better, even if it will Nx the tokens you need to achieve something, though in general it will be cheaper than N human reviewers (which you might not have). Obviously you shouldn't forget about traditional tooling for formatting and linting, as well as static code analysis and having test coverage that approaches 100%. It might be annoying to do manually, but AI has no issues with refactoring code to make it more testable and eventually will catch some issues that way. It's never going to be perfect in the 1st attempt. > Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. This is far too vague though and will never be good, even sans AI. When it comes to AI, it will nitpick the fuck out of the codebase if you ask it to do and sometimes jump around between different approaches because neither is actually a good fit for the problem space (there might not be a good fit at all, just various tradeoffs). If you still ask it to find issues and there's nothing obvious, it will just make shit up in pursuit of being useful (RLHF). When it comes to people, you will get various standards, from "It looks like Java, ship it" to "You should rework a quarter of your codebase because I read about this one approach in an authoritatively written book that you should also follow because I view it as dogma and will hold back your merge until it all works exactly like I want it to." (you get all sorts of people and personalities). In my experience other people are no panacea either, nor is writing code all by myself. Fuck it, I'll take anything and everything to help me ship stuff that's good enough and on time (even if some/most? deadlines within the industry are made up). I'd argue that producing something that would pass most critique and could be considered "good code" (not "good enough") or even more broadly a "good product" might take about an order of magnitude more effort than most people and organizations actually can, or can budget for.
- mosura 2mo ago
  > Adversarial review loops of N parallel agents looking at whatever characteristics you care about will make it better Especially if different model providers. It becomes more like having team members that see things slightly differently.
  - ModernMech 2mo ago
    It's like a variant of the halting problem though: given N agents in a review loop reviewing a codebase, will it ever terminate and say the thing is done and bug free? It seems to me just from @codex review, given a codebase of any appreciable size, if you ask the agent to find things to fix, it'll find things to fix. And despite N agents agreeing that some code is ready to ship, I've still sat down to try it and nothing actually works as advertised. So the need for better verification tools that are not AI is very urgent. Because the AI can be told to write a thing without mistakes, it does write code that compiles, N-agent AI review process fixes bugs and eventually the code is approved, and then run against an extensive suite of tests to prove certain functionality which all pass... and yet it still can be the case that nothing actually works in production.
    - KronisLV 2mo ago
      > will it ever terminate and say the thing is done and bug free? Will human reviewers, if you ask them to find more bugs? Will it be because there really are no bugs, or because they got tired and just can't see any? How would you even figure out when the thing is done and is bug free? There have been bugs that have been in codebases for years and have only been discovered recently. Would you trust static analysis tools that give you an all clear, when they themselves cannot encode the above desired state fully? Seems like you'd need formal proofs or something, but at that point also for the frameworks and libraries, database, networking stack and the whole damn OS. I'm all for good tools, just saying that we'll need a whole bunch of those.
      - ModernMech 2mo ago
        Yes, I've often mused that the halting problem should have corollary for determining when a faculty meeting will end, if at all.
geraneum 2mo ago
Then ask it to fix it. When “fixed”, ask the same question again and you’ll get a similar response again!
- tim-projects 2mo ago
  Is the right answer. I dunno what drivel the rest of the commentators are on about.
budsniffer952 2mo ago
Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfully every day that, actually, it's just a mirage?
- deleted 2mo ago
  [deleted]
- Kiro 2mo ago
  Yeah, it's baffling. I can't relate to these statements at all. What are people doing? Surely the smart people of HN would have been able to figure this out a long time ago. I also don't find these people in real life. Even the most junior developers I know are able to navigate this without creating this supposed mess.
  - gregoryl 2mo ago
    Its a bit unkind to talk like this - the obvious and equally unproductive response is to question if you are really as good as you think you are. Are those junior developers not making a mess, or do you lack the insight to see it?
    - lelele 2mo ago
      This.
    - anal_reactor 2mo ago
      Big codebases tend to become a mess anyway so if your company has experience dealing with shit, the fact that now shit is AI-generated doesn't substantially change anything. And I'm not joking. Imagine it this way - managing a group of skilled professionals is a completely different skill from managing a bunch of alcoholics doing a minimum-wage job, and sometimes the latter situation is just the reality you find yourself in.
faangguyindia 2mo ago
>? I'm selling it for $1million dollars can it meet that standard. but people have sold terrible codebases for more than a million dollar.
- mosura 2mo ago
  Yeah the financial value of a codebase is in successfully solving a problem, which is independent of code quality. It is a painful lesson for many.
  - cube00 2mo ago
    > Yeah the financial value of a codebase is in successfully solving a problem It doesn't even need do it successfully, plenty of slow buggy codebases that are raking in huge amounts of money every day. I knew it was bad when I started getting tickets to "improve the skeleton loader" because its displayed for so long the project owner had time to contemplate changes to it.
    - gmueckl 2mo ago
      The success or value of a solution isn't measured by its quality, but by its utility to the customer. It's a harsh lesson for anyone who wants to build quality.
      - mosura 2mo ago
        And even worse “customer utility” isn’t real but perceived. It is more important to make the buyer feel the solution helps than anything else.
  - tripleee 2mo ago
    code quality helps you get to the final successful solution, keep it there and iterate on it faster and with less bugs. It's not just pretty indentation but yeah sure no value
sschueller 2mo ago
OpenClaw sold for how much?...
matheusmoreira 2mo ago
How many millions of dollars should Oracle database sell for? https://news.ycombinator.com/item?id=18442941 https://news.ycombinator.com/item?id=18442941
xnx 2mo ago
I get better result when I say $1 billion dollars. /s

KronisLV 2mo ago

> Then cry as the ai reveals that it didn't actually do anything close to what it said it did. If using AI to generate code, you told it generate some code, so it did. No amount of "You are an expert developer" or "Make no mistakes" will change the fact that it just generates tokens and has a limited thinking budget. Adversarial review loops of N parallel agents looking at whatever characteristics you care about will make it better, even if it will Nx the tokens you need to achieve something, though in general it will be cheaper than N human reviewers (which you might not have). Obviously you shouldn't forget about traditional tooling for formatting and linting, as well as static code analysis and having test coverage that approaches 100%. It might be annoying to do manually, but AI has no issues with refactoring code to make it more testable and eventually will catch some issues that way. It's never going to be perfect in the 1st attempt. > Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. This is far too vague though and will never be good, even sans AI. When it comes to AI, it will nitpick the fuck out of the codebase if you ask it to do and sometimes jump around between different approaches because neither is actually a good fit for the problem space (there might not be a good fit at all, just various tradeoffs). If you still ask it to find issues and there's nothing obvious, it will just make shit up in pursuit of being useful (RLHF). When it comes to people, you will get various standards, from "It looks like Java, ship it" to "You should rework a quarter of your codebase because I read about this one approach in an authoritatively written book that you should also follow because I view it as dogma and will hold back your merge until it all works exactly like I want it to." (you get all sorts of people and personalities). In my experience other people are no panacea either, nor is writing code all by myself. Fuck it, I'll take anything and everything to help me ship stuff that's good enough and on time (even if some/most? deadlines within the industry are made up). I'd argue that producing something that would pass most critique and could be considered "good code" (not "good enough") or even more broadly a "good product" might take about an order of magnitude more effort than most people and organizations actually can, or can budget for.

mosura 2mo ago
> Adversarial review loops of N parallel agents looking at whatever characteristics you care about will make it better Especially if different model providers. It becomes more like having team members that see things slightly differently.
- ModernMech 2mo ago
  It's like a variant of the halting problem though: given N agents in a review loop reviewing a codebase, will it ever terminate and say the thing is done and bug free? It seems to me just from @codex review, given a codebase of any appreciable size, if you ask the agent to find things to fix, it'll find things to fix. And despite N agents agreeing that some code is ready to ship, I've still sat down to try it and nothing actually works as advertised. So the need for better verification tools that are not AI is very urgent. Because the AI can be told to write a thing without mistakes, it does write code that compiles, N-agent AI review process fixes bugs and eventually the code is approved, and then run against an extensive suite of tests to prove certain functionality which all pass... and yet it still can be the case that nothing actually works in production.
  - KronisLV 2mo ago
    > will it ever terminate and say the thing is done and bug free? Will human reviewers, if you ask them to find more bugs? Will it be because there really are no bugs, or because they got tired and just can't see any? How would you even figure out when the thing is done and is bug free? There have been bugs that have been in codebases for years and have only been discovered recently. Would you trust static analysis tools that give you an all clear, when they themselves cannot encode the above desired state fully? Seems like you'd need formal proofs or something, but at that point also for the frameworks and libraries, database, networking stack and the whole damn OS. I'm all for good tools, just saying that we'll need a whole bunch of those.
    - ModernMech 2mo ago
      Yes, I've often mused that the halting problem should have corollary for determining when a faculty meeting will end, if at all.

geraneum 2mo ago

Then ask it to fix it. When “fixed”, ask the same question again and you’ll get a similar response again!

tim-projects 2mo ago
Is the right answer. I dunno what drivel the rest of the commentators are on about.

budsniffer952 2mo ago

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfully every day that, actually, it's just a mirage?

deleted 2mo ago
[deleted]
Kiro 2mo ago
Yeah, it's baffling. I can't relate to these statements at all. What are people doing? Surely the smart people of HN would have been able to figure this out a long time ago. I also don't find these people in real life. Even the most junior developers I know are able to navigate this without creating this supposed mess.
- gregoryl 2mo ago
  Its a bit unkind to talk like this - the obvious and equally unproductive response is to question if you are really as good as you think you are. Are those junior developers not making a mess, or do you lack the insight to see it?
  - lelele 2mo ago
    This.
  - anal_reactor 2mo ago
    Big codebases tend to become a mess anyway so if your company has experience dealing with shit, the fact that now shit is AI-generated doesn't substantially change anything. And I'm not joking. Imagine it this way - managing a group of skilled professionals is a completely different skill from managing a bunch of alcoholics doing a minimum-wage job, and sometimes the latter situation is just the reality you find yourself in.

faangguyindia 2mo ago

>? I'm selling it for $1million dollars can it meet that standard. but people have sold terrible codebases for more than a million dollar.

mosura 2mo ago
Yeah the financial value of a codebase is in successfully solving a problem, which is independent of code quality. It is a painful lesson for many.
- cube00 2mo ago
  > Yeah the financial value of a codebase is in successfully solving a problem It doesn't even need do it successfully, plenty of slow buggy codebases that are raking in huge amounts of money every day. I knew it was bad when I started getting tickets to "improve the skeleton loader" because its displayed for so long the project owner had time to contemplate changes to it.
  - gmueckl 2mo ago
    The success or value of a solution isn't measured by its quality, but by its utility to the customer. It's a harsh lesson for anyone who wants to build quality.
    - mosura 2mo ago
      And even worse “customer utility” isn’t real but perceived. It is more important to make the buyer feel the solution helps than anything else.
- tripleee 2mo ago
  code quality helps you get to the final successful solution, keep it there and iterate on it faster and with less bugs. It's not just pretty indentation but yeah sure no value

sschueller 2mo ago

OpenClaw sold for how much?...

matheusmoreira 2mo ago

How many millions of dollars should Oracle database sell for? https://news.ycombinator.com/item?id=18442941 https://news.ycombinator.com/item?id=18442941

xnx 2mo ago

I get better result when I say $1 billion dollars. /s