4 ms·

From my experience using ai, if you don't write a really precise description of your initial requirements in the initial prompt, and it it doesn't one-shot the

From my experience using ai, if you don't write a really precise description of your initial requirements in the initial prompt, and it it doesn't one-shot the answer, I don't bother asking it to fix the mistake. Unless you're using a LLM with really long context, it's prone that some context loss will happen soner or later - when that happens, pointing errors that dropped out of the context will just result in repeated or garbage output.

theshrike79 2y ago
This is why platforms(?) like Claude Code, Cursor and Windsurf are essential. Claude Code forces you to create a CLAUDE.md to direct how it works, with Cursor you can (and should) write Cursor Rules. The difference with a good spec + AI vs just vibe coding from scratch is like night and day.
- larusso 2y ago
  I don‘t use these tools daily because I have a hard time to commit to these workflows. But do I need to do this setup once, once per project or every time for one off things I might code?
  - theshrike79 2y ago
    You really don't need these for tiny one-off scripts, but they're essential for larger projects where the whole application can't fit into the LLM context at once. Basically they're just a markdown files where you write what the project is about, where everything is located and things like that. Pretty much something you'd need to have for another human to bring them up to speed. An example snippet from the project I just happened to have open: ## Code Structure - All import processors go under the `cmd/` directory - The internal/ directory contains common utility functions, use it when possible. Add to it when necessary. - Each processor has its own subdirectory and Go package ## Implementation Requirements - Maintain consistent Go style and idiomatic patterns - Follow the existing architectural patterns - Each data source processor should be implemented as a separate command This way the LLM (Cursor in this project) knows to, for example, check the internal/ directory for common utils. And if it finds duplication, it'll automatically generate any common functions in there. It's a way to add guidelines/rails for projects, if you don't add anything the LLM will just pick whatever. It may even change style depending on what's being implemented. In this project the Goodreads processor was 100% different from the Steam processor. A human would've seen the similarities and followed the same style, but the LLM can't do that without help.
- dingnuts 2y ago
  why would I pay for the advanced features when I haven't been impressed with the free features? in fact Claude 3.5, which is what is available, is a nearly worthless product, with value comparable to a free search engine, and not even a very good one. It is usually incorrect and frequently in subtle ways that will cost me a lot of time. pro AI people sound like someone with an expensive addiction trying to justify it. the free product is bad, so I just need to pay to see the light? Why would Anthropic let me use a model for free that is going to make me more skeptical of their paid offerings unless it is pretty similar to the paid ones and they think it's good? Just read the manual and write the code yourself. These toys are a distraction.
  - johnisgood 2y ago
    Free version of Claude sucks because the prompts get too long very quickly. I have the Pro version and I have completed many tasks successfully. (I know what I was doing and had a great understanding of what is going on).
  - NeutralCrane 2y ago
    Like many tools, there is some user skill required. Certainly there are situations where AI assistants won’t help much, but if every single attempt you’ve made to use an AI coding assistant has been “useless”, you are either working in a very niche area or, perhaps more likely, it is user error on your own part. There are plenty of people who are way too high on the current abilities of AI, but I find that the “AI is useless crowd” to be equally ridiculous. It reminds me of early in my career working in statistics where the company I joined out of grad school was justifiably looking to move out of SAS and start working in R and Python. Many were enthusiastic about the change and quickly saw the benefit, but there were some who were too entrenched in their previous way of working, and insisted that there was no benefit to changing, they could do anything required in SAS, and stubbornly refused to entertain the idea that there was a benefit to be gained by learning a new skill. You needn’t become an AI cultist. But with the number of people who are getting at least some benefit to using AI coding assistants, if you are finding it to be worthless in your personal experience, it may be worth stepping back and considering if there is something wrong with how you are trying to utilize it.
derekp7 2y ago
What I do is go back through the conversation history, select the response that has the somewhat working code, then submit a prompt with what I want changed. By selectively including context and adjusting temperature, top_p/k and sometimes swapping model or system prompt for a given query will give better results. Combine this with repeating the query multiple times with that same context, then select which result is the best and move on.

theshrike79 2y ago

This is why platforms(?) like Claude Code, Cursor and Windsurf are essential. Claude Code forces you to create a CLAUDE.md to direct how it works, with Cursor you can (and should) write Cursor Rules. The difference with a good spec + AI vs just vibe coding from scratch is like night and day.

larusso 2y ago
I don‘t use these tools daily because I have a hard time to commit to these workflows. But do I need to do this setup once, once per project or every time for one off things I might code?
- theshrike79 2y ago
  You really don't need these for tiny one-off scripts, but they're essential for larger projects where the whole application can't fit into the LLM context at once. Basically they're just a markdown files where you write what the project is about, where everything is located and things like that. Pretty much something you'd need to have for another human to bring them up to speed. An example snippet from the project I just happened to have open: ## Code Structure - All import processors go under the `cmd/` directory - The internal/ directory contains common utility functions, use it when possible. Add to it when necessary. - Each processor has its own subdirectory and Go package ## Implementation Requirements - Maintain consistent Go style and idiomatic patterns - Follow the existing architectural patterns - Each data source processor should be implemented as a separate command This way the LLM (Cursor in this project) knows to, for example, check the internal/ directory for common utils. And if it finds duplication, it'll automatically generate any common functions in there. It's a way to add guidelines/rails for projects, if you don't add anything the LLM will just pick whatever. It may even change style depending on what's being implemented. In this project the Goodreads processor was 100% different from the Steam processor. A human would've seen the similarities and followed the same style, but the LLM can't do that without help.
dingnuts 2y ago
why would I pay for the advanced features when I haven't been impressed with the free features? in fact Claude 3.5, which is what is available, is a nearly worthless product, with value comparable to a free search engine, and not even a very good one. It is usually incorrect and frequently in subtle ways that will cost me a lot of time. pro AI people sound like someone with an expensive addiction trying to justify it. the free product is bad, so I just need to pay to see the light? Why would Anthropic let me use a model for free that is going to make me more skeptical of their paid offerings unless it is pretty similar to the paid ones and they think it's good? Just read the manual and write the code yourself. These toys are a distraction.
- johnisgood 2y ago
  Free version of Claude sucks because the prompts get too long very quickly. I have the Pro version and I have completed many tasks successfully. (I know what I was doing and had a great understanding of what is going on).
- NeutralCrane 2y ago
  Like many tools, there is some user skill required. Certainly there are situations where AI assistants won’t help much, but if every single attempt you’ve made to use an AI coding assistant has been “useless”, you are either working in a very niche area or, perhaps more likely, it is user error on your own part. There are plenty of people who are way too high on the current abilities of AI, but I find that the “AI is useless crowd” to be equally ridiculous. It reminds me of early in my career working in statistics where the company I joined out of grad school was justifiably looking to move out of SAS and start working in R and Python. Many were enthusiastic about the change and quickly saw the benefit, but there were some who were too entrenched in their previous way of working, and insisted that there was no benefit to changing, they could do anything required in SAS, and stubbornly refused to entertain the idea that there was a benefit to be gained by learning a new skill. You needn’t become an AI cultist. But with the number of people who are getting at least some benefit to using AI coding assistants, if you are finding it to be worthless in your personal experience, it may be worth stepping back and considering if there is something wrong with how you are trying to utilize it.

derekp7 2y ago

What I do is go back through the conversation history, select the response that has the somewhat working code, then submit a prompt with what I want changed. By selectively including context and adjusting temperature, top_p/k and sometimes swapping model or system prompt for a given query will give better results. Combine this with repeating the query multiple times with that same context, then select which result is the best and move on.