4 ms·

> i'm not sure why people expect agents to one shot everything to perfection with just a prompt. because that's how agents are marketed.

by 0x457 2mo ago

uproarchat 2mo ago
I've never seen model providers marketing like that. What examples have you seen?
- Lalabadie 2mo ago
  I don't really think they advertise "Create your app idea in one weekend night" and assume the general public will mentally add "... but hire an experienced developer to supervise the process".
- 0x457 2mo ago
  Literally any coding agent marking material: - https://cognition.com/ https://cognition.com/ - https://openai.com/index/introducing-the-codex-app/ https://openai.com/index/introducing-the-codex-app/ - https://www.anthropic.com/news/claude-3-7-sonnet https://www.anthropic.com/news/claude-3-7-sonnet anthropic specifically brags about how good claude code is every annoucement of a new model. I will surrender that none of them claim its "to perfection", but IMO its implied because no one would claim that their model one-shots any issue to dog shit quality.
  - hombre_fatal 2mo ago
    Seems like motte and bailey fallacy. They say their models are good (the motte), therefore their models must one-shot everything to perfection (the bailey). Besides, other people's claims about something doesn't give you license to abandon all critical thinking. Though it's evident they don't claim what you say they are.
    - 0x457 2mo ago
      That's irrelevant. Questions was why people assume somthing, and answer is because that's how it advertised. To be clear that's not what I'm thinking, even Fable 5 produces some hilariously bad results under some conditions and sonnet 5 produced great results under others.
      - hombre_fatal 2mo ago
        > because that's how it advertised. But you didn't provide the evidence for that. You shared some links and then admitted they didn't claim it. It kinda seems like "because I think they're a little too positive about their product, I can set my expectations to anything I want and la-la-la it's their fault." And I don't see the problem with agents building test scaffolding as they go. It might be too defensive at times, like testing a shell script you don't run often, but big deal. It's kinda cool imo, and it's trivial to make it stop.
  - infinite_spin 2mo ago
    - https://openai.com/index/introducing-the-codex-app/ https://openai.com/index/introducing-the-codex-app/ no where does this document suggest that codex can "one shot everything to perfection with just a prompt". It describes using a prompt plus agent skills (which are essentially many other prompts) to develop a playable game.. nothing about it being perfect or anything more than being in a playable state.
    - joshribakoff 2mo ago
      Oh please, you sound so disingenuous, the claim wasn’t that the documents contained a specific phrase. You’re moving the goal posts. Their name is literally a play on anthropomorphising the models, such as… the ceo going on tv shows and repeatedly saying the models may be conscious and they may start nuclear wars, etc.
      - infinite_spin 2mo ago
        I'm content with my response.
- mrheosuper 2mo ago
  I've seen a lot of cursor app, about some PO that has an idea for an app in the morning, then asking her agent to make it when her commuting, when arrive at office the app is done
- onion2k 2mo ago
  Claude has a /goal function that explicitly says it'll carry on working until it's done what you prompted it to do. That's exactly what a 'AI will zero-shot anything' believer is looking for. Behind the scenes it's really multi-shotting with generated prompts, but the user won't care.
deleted 2mo ago
[deleted]
vanuatu 2mo ago
we should exercise critical thinking then heaps of people on this site expect them to be omnipotent then claim it’s fake when it doesn’t read minds
- joshribakoff 2mo ago
  You’re right — people really should think critically, but the issue remains, that many do not.
- bigstrat2003 2mo ago
  I think it's perfectly fair to evaluate the tools based on how well they live up to the hype that is being pumped out by the sellers of said tools. If they want us to compare their products to a more measured, reasonable take then they can advertise them as that.
  - sfn42 2mo ago
    While some people are busy bickering about this, the rest of us are using these awesome new tools to get more work done in less time with higher quality than ever. I don't care what the company claims, I just use the tool the way I want to. I work very closely with the AI. I'll tell it to plan a change, review the plan, then execute. Then I'll test the changes and have it fix whatever I'm not happy with one thing at a time. I'll specify in detail both what to do and loosely describe how to do it or if I'm not sure I'll ask it to plan the change then review the plan and ask for changes if I want them etc. I also review my own PRs before I submit them to colleagues. This way I maintain full control of everything, it just saves me hours of googling, planning and typing code - which I do miss a bit but I can't really justify writing code myself when I can achieve the same thing just by loosely describing my idea instead. It also saves a lot of time debugging, I think I'm generally a pretty good programmer but the AI makes fewer mistakes than me. It'll often catch some logic error I made during planning and suggest a good alternative. A lot of developers seem to give up control entirely and then complain that they're no longer in control. Trying for that 10-100x speedup doing weeks of work in a day. I'm happy doing one week of work in a day. There's a limit to how much I can oversee without compromising quality.
- 2mo ago
addandsubtract 2mo ago
Not just marketed, but also benchmarked (and benchmaxxed).

> i'm not sure why people expect agents to one shot everything to perfection with just a prompt. because that's how agents are marketed.

by 0x457 2mo ago

> i'm not sure why people expect agents to one shot everything to perfection with just a prompt. because that's how agents are marketed.

uproarchat 2mo ago

I've never seen model providers marketing like that. What examples have you seen?

Lalabadie 2mo ago
I don't really think they advertise "Create your app idea in one weekend night" and assume the general public will mentally add "... but hire an experienced developer to supervise the process".
0x457 2mo ago
Literally any coding agent marking material: - https://cognition.com/ https://cognition.com/ - https://openai.com/index/introducing-the-codex-app/ https://openai.com/index/introducing-the-codex-app/ - https://www.anthropic.com/news/claude-3-7-sonnet https://www.anthropic.com/news/claude-3-7-sonnet anthropic specifically brags about how good claude code is every annoucement of a new model. I will surrender that none of them claim its "to perfection", but IMO its implied because no one would claim that their model one-shots any issue to dog shit quality.
- hombre_fatal 2mo ago
  Seems like motte and bailey fallacy. They say their models are good (the motte), therefore their models must one-shot everything to perfection (the bailey). Besides, other people's claims about something doesn't give you license to abandon all critical thinking. Though it's evident they don't claim what you say they are.
  - 0x457 2mo ago
    That's irrelevant. Questions was why people assume somthing, and answer is because that's how it advertised. To be clear that's not what I'm thinking, even Fable 5 produces some hilariously bad results under some conditions and sonnet 5 produced great results under others.
    - hombre_fatal 2mo ago
      > because that's how it advertised. But you didn't provide the evidence for that. You shared some links and then admitted they didn't claim it. It kinda seems like "because I think they're a little too positive about their product, I can set my expectations to anything I want and la-la-la it's their fault." And I don't see the problem with agents building test scaffolding as they go. It might be too defensive at times, like testing a shell script you don't run often, but big deal. It's kinda cool imo, and it's trivial to make it stop.
- infinite_spin 2mo ago
  - https://openai.com/index/introducing-the-codex-app/ https://openai.com/index/introducing-the-codex-app/ no where does this document suggest that codex can "one shot everything to perfection with just a prompt". It describes using a prompt plus agent skills (which are essentially many other prompts) to develop a playable game.. nothing about it being perfect or anything more than being in a playable state.
  - joshribakoff 2mo ago
    Oh please, you sound so disingenuous, the claim wasn’t that the documents contained a specific phrase. You’re moving the goal posts. Their name is literally a play on anthropomorphising the models, such as… the ceo going on tv shows and repeatedly saying the models may be conscious and they may start nuclear wars, etc.
    - infinite_spin 2mo ago
      I'm content with my response.
mrheosuper 2mo ago
I've seen a lot of cursor app, about some PO that has an idea for an app in the morning, then asking her agent to make it when her commuting, when arrive at office the app is done
onion2k 2mo ago
Claude has a /goal function that explicitly says it'll carry on working until it's done what you prompted it to do. That's exactly what a 'AI will zero-shot anything' believer is looking for. Behind the scenes it's really multi-shotting with generated prompts, but the user won't care.

deleted 2mo ago

[deleted]

vanuatu 2mo ago

we should exercise critical thinking then heaps of people on this site expect them to be omnipotent then claim it’s fake when it doesn’t read minds

joshribakoff 2mo ago
You’re right — people really should think critically, but the issue remains, that many do not.
bigstrat2003 2mo ago
I think it's perfectly fair to evaluate the tools based on how well they live up to the hype that is being pumped out by the sellers of said tools. If they want us to compare their products to a more measured, reasonable take then they can advertise them as that.
- sfn42 2mo ago
  While some people are busy bickering about this, the rest of us are using these awesome new tools to get more work done in less time with higher quality than ever. I don't care what the company claims, I just use the tool the way I want to. I work very closely with the AI. I'll tell it to plan a change, review the plan, then execute. Then I'll test the changes and have it fix whatever I'm not happy with one thing at a time. I'll specify in detail both what to do and loosely describe how to do it or if I'm not sure I'll ask it to plan the change then review the plan and ask for changes if I want them etc. I also review my own PRs before I submit them to colleagues. This way I maintain full control of everything, it just saves me hours of googling, planning and typing code - which I do miss a bit but I can't really justify writing code myself when I can achieve the same thing just by loosely describing my idea instead. It also saves a lot of time debugging, I think I'm generally a pretty good programmer but the AI makes fewer mistakes than me. It'll often catch some logic error I made during planning and suggest a good alternative. A lot of developers seem to give up control entirely and then complain that they're no longer in control. Trying for that 10-100x speedup doing weeks of work in a day. I'm happy doing one week of work in a day. There's a limit to how much I can oversee without compromising quality.
2mo ago

addandsubtract 2mo ago

Not just marketed, but also benchmarked (and benchmaxxed).