7 ms·

Could be that some people are not familiar with the limitations of LLMs and what to expect from them, and could benefit from learning how to properly do prompti

by rvnx 1y ago

Could be that some people are not familiar with the limitations of LLMs and what to expect from them, and could benefit from learning how to properly do prompting, before outright rejecting them. The LLMs are sometimes frustrating, but over time you learn how to drive them properly, exactly like when you learn how to use the right keywords on Google. Also, considering he is a free user, the author might be using one the default models ("auto" -> o4-mini or gpt-4o, rarely Claude Sonnet/Opus 3) in Cursor -> Both of them are rather weak, especially with the automatic context splitting that Cursor does to save money Claude Code + Opus 4 would likely yield better results here.

rednafi 1y ago
> Also, considering he is a free user, the author might be using one the default models ("auto" -> o4-mini or gpt-4o, rarely Claude Sonnet/Opus 3) in Cursor I wonder where you got the free user info.
- rvnx 1y ago
  Possibly, though the context strongly suggests someone discovering Cursor. It's also possible that someone shared their account with him and that could explain the "30-minute thinking time" (the free slow requests).
Chinjut 1y ago
Just use the right prompt. Just use the right model. Just roll the dice correctly.
- dinfinity 1y ago
  This is too dismissive. There is a massive difference in outputted code quality between models. The recent Google Gemini 2.5 models have dethroned the Anthropic ones. OpenAI o3 is the only other one that is even worth considering; all the rest of the OpenAI models are trash in comparison. In addition to that we've also seen that the way you prompt (amount of use of expert language), what context you provide through instructions, and tool use make a huge difference on the outcome. At this point, if your coding experience with LLMs sucks I'd say there is an 80% chance that you're just doing it wrong.
  - creesch 1y ago
    The differences can't be that massive given that the hype already made many of these promises well before these models were ever a thing. Basically you are just adding to it with "You just need to use the latest model, anything before that is trash". Ignoring that the same was said just a few months prior when those models were cutting edge.
    - dinfinity 1y ago
      > The differences can't be that massive given that the hype already made many of these promises well before these models were ever a thing. I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before. > Basically you are just adding to it with "You just need to use the latest model, anything before that is trash". That's not what I said. Don't put words into my mouth. I said that the older models are trash in comparison, not that they are trash. The older models require more work in prompting to get decent results. > Ignoring that the same was said just a few months prior when those models were cutting edge. You are conveniently ignoring the other parts I mentioned to make my claim. Here: "In addition to that we've also seen that the way you prompt (amount of use of expert language), what context you provide through instructions, and tool use make a huge difference on the outcome."
      - creesch 1y ago
        > I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before. Alright? What you replied to and the context of this entire thread is about promises that have been made for a while now. In fact, we are approaching the point where we can safely talk about years of hype now. For reference, I am using the gpt-4 release as a significant ramp-up in the hype around LLMs. > That's not what I said. Don't put words into my mouth. That might not have been your intention. But, again, given the context of where you are replying it does read like. Even with the qualifier of "in comparison". Like it or not, your comment does add extra qualifiers to the list. Look, I am not saying that there isn't any progress being made here. I also agree that LLMs can be useful tools as part of a developer toolkit. What I personally don't agree with is that they can do the same job, even less so in real world scenarios. Even the latest models, including Gemini 2.5 and 03 struggle with moderately complex code base. And yes, the argument is always to let them work on small isolated bits of code. Or that if your requirements are tight enough they produce very good code. Which is entirely true, but I also envy the developers who work in structured environments where their code base is that clean and requirements that well-defined. So, in my experience, coding with these models still sucks. Using them as interactive rubber duckies, replacements for some of the things I used to spend hours googling, debugging small snippets of code, etc. Sure, there they are very useful tools to me. But, to me, that is not coding with LLMs. That is having LLMs available as a tool whenever I need them.

by rvnx 1y ago

rednafi 1y ago

> Also, considering he is a free user, the author might be using one the default models ("auto" -> o4-mini or gpt-4o, rarely Claude Sonnet/Opus 3) in Cursor I wonder where you got the free user info.

rvnx 1y ago
Possibly, though the context strongly suggests someone discovering Cursor. It's also possible that someone shared their account with him and that could explain the "30-minute thinking time" (the free slow requests).

Chinjut 1y ago

Just use the right prompt. Just use the right model. Just roll the dice correctly.

dinfinity 1y ago
This is too dismissive. There is a massive difference in outputted code quality between models. The recent Google Gemini 2.5 models have dethroned the Anthropic ones. OpenAI o3 is the only other one that is even worth considering; all the rest of the OpenAI models are trash in comparison. In addition to that we've also seen that the way you prompt (amount of use of expert language), what context you provide through instructions, and tool use make a huge difference on the outcome. At this point, if your coding experience with LLMs sucks I'd say there is an 80% chance that you're just doing it wrong.
- creesch 1y ago
  The differences can't be that massive given that the hype already made many of these promises well before these models were ever a thing. Basically you are just adding to it with "You just need to use the latest model, anything before that is trash". Ignoring that the same was said just a few months prior when those models were cutting edge.
  - dinfinity 1y ago
    > The differences can't be that massive given that the hype already made many of these promises well before these models were ever a thing. I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before. > Basically you are just adding to it with "You just need to use the latest model, anything before that is trash". That's not what I said. Don't put words into my mouth. I said that the older models are trash in comparison, not that they are trash. The older models require more work in prompting to get decent results. > Ignoring that the same was said just a few months prior when those models were cutting edge. You are conveniently ignoring the other parts I mentioned to make my claim. Here: "In addition to that we've also seen that the way you prompt (amount of use of expert language), what context you provide through instructions, and tool use make a huge difference on the outcome."
    - creesch 1y ago
      > I'm not talking about other people's incorrect promises _and_ I mentioned a number of things in which proper usage today is different from what people were doing before. Alright? What you replied to and the context of this entire thread is about promises that have been made for a while now. In fact, we are approaching the point where we can safely talk about years of hype now. For reference, I am using the gpt-4 release as a significant ramp-up in the hype around LLMs. > That's not what I said. Don't put words into my mouth. That might not have been your intention. But, again, given the context of where you are replying it does read like. Even with the qualifier of "in comparison". Like it or not, your comment does add extra qualifiers to the list. Look, I am not saying that there isn't any progress being made here. I also agree that LLMs can be useful tools as part of a developer toolkit. What I personally don't agree with is that they can do the same job, even less so in real world scenarios. Even the latest models, including Gemini 2.5 and 03 struggle with moderately complex code base. And yes, the argument is always to let them work on small isolated bits of code. Or that if your requirements are tight enough they produce very good code. Which is entirely true, but I also envy the developers who work in structured environments where their code base is that clean and requirements that well-defined. So, in my experience, coding with these models still sucks. Using them as interactive rubber duckies, replacements for some of the things I used to spend hours googling, debugging small snippets of code, etc. Sure, there they are very useful tools to me. But, to me, that is not coding with LLMs. That is having LLMs available as a tool whenever I need them.