3 ms·
> Even if I prompt with security as the top priority LLMs do really poorly with general statements like that, so not sure it's unexpected. If you put "Make sur
by diggan 1y ago
> Even if I prompt with security as the top priority
LLMs do really poorly with general statements like that, so not sure it's unexpected. If you put "Make sure to make it production worthy", you'll get as many different answers as there are programmers, because not even us human programmers agree what that really means.
Same for "Make sure security is the top priority", most programmers would understand that differently. If you instead spell out exactly what behavior you expect from something like that (so "Make sure there are no XSS's", "Make sure users can't bypass authentication" and so on), you'll get a much higher probability it'll manage to follow those.
- ednite 1y agoTotally agree. I was simplifying for discussion’s sake, but yeah, I learned the hard way that vague prompts will lead you that rabbit hole. If you’re not crystal clear, you get everything but what you actually wanted. These days, I make sure every “t” is crossed and every “i” dotted when giving instructions. Good point, definitely a lesson worth repeating.
- kalleboo 1y agoAlthough the promise of the latest generation of "thinking" models is that they're supposed to do that themselves - in the internal chat thread in their "thought" process they're supposed to go "what does secure mean for a web app", list those items (which it can already easily do if you prompt it separately) and then go through that list