4 ms·
This nonsense illustrates the most typical way that people misunderstand the word "engineering" in a software context. Software engineering and prompt engineeri
by ilaksh 4y ago
This nonsense illustrates the most typical way that people misunderstand the word "engineering" in a software context. Software engineering and prompt engineering is not about the self-proclaimed level of rigor or formality that you apply. It's about the actual knowledge and processes used and especially their _effectiveness_ as measured in closed feedback loops.
But the starting point for this is that the term "prompt engineering" is an obvious exaggeration that people are using to promote a skill set which is real and very useful but a big stretch to describe as a whole new engineering discipline.
Regardless of what you call it, like software engineering, it really is a process of trial and error for the most part. With the capabilities of the latest OpenAI models, you should be aiming for a level of generality where most tasks are not going to have a simple answer that you can automatically check to create an accuracy score. EDIT: after thinking about it, there certainly are tasks that you could check for specific answers to create an accuracy score, but I still think it would make more sense in most cases to instead spend time iterating on user feedback rather than trying to think of comprehensive test cases on your own. There are a few things to know, such as the idea of providing examples, the necessary context, and telling the model to work step-by-step.
Actually I would say that there are two major things that could be improved in the engineering described in this article related to actually closing the feedback loops he mentions. He really should at least mention the possibility of coming up with a new prompt candidate after he was done with the first round of tests and also after the users found some problem cases.
The main thing is to close the feedback loops.
- manojlds 4y agoI think prompt engineering is social engineering but for AI.
- dmix 4y agoOf course, LLMs are like toddlers, prompt engineering is holding their hand so they get from A->B as best as possible. It's providing important context to frame output, guiding it along to "think" out it's processes, and breaking bigger problems down into smaller pieces. Essentially turning prompts into a series of smaller feedback loops - rather than dumping a big query and hoping the baseline toddler is smart enough to figure it out. It's just about knowing the person you're interacting with extremely well, knowing their quirks and what motivates them, and carefully guiding them along, so they do what you want. Much like social engineering.
- fennecfoxy 4y ago* Toddlers who've just ingested the AI equivalent of a bucket of sugar
- LegitShady 4y agowriters are now word engineers, artists are image engineers, and politicians are now bullshit engineers. Prompt engineering is just people putting effort into studying the behaviour of ML models and how input affects the output. They're more like ML psychologists than engineers. Calling themselves engineers just makes them feel better about being glorified prompt testers.
- H8crilA 4y agoTIL that developing something like the Google Search ranking function is not engineering.
- LegitShady 4y agopsychologists and sociologists use statistics to evaluate their results, does that make them engineers too?
- kgwgk 4y agoEngineers of the human soul: https://en.wikipedia.org/wiki/Engineers_of_the_human_soul https://en.wikipedia.org/wiki/Engineers_of_the_human_soul
- LegitShady 4y agoso you're taking classification advice from joseph stalin? Maybe reconsider whether thats applicable to the situation or not.
- lucubratory 4y agoThat's beautiful, I'd never heard of it before. Thank you.
- davidthewatson 4y agoNo, but it may mean that prompt engineering is a human science on the spectrum from phenomenology to scientism. ¯\_(ツ)_/¯
- cfn 4y agoI believe it is a bit too much to call the article nonsense. The process described mirrors what we have been doing in Machine Learning for a long time: You setup a training set and validation set, put it through system under test and draw conclusions from the statistical analysis of the results.
- deleted 4y ago[deleted]
- stevev 4y agoThe appropriate name for this is Prompt Designer, Model Tuners or Model Tuning imo.
- fennecfoxy 4y agoLmao, nobody's equating a prompt engineer with a civil or rail etc engineer. Stop being pedantic "x engineer" has become a common phrase to describe someone who works on something related to the tech industry, the word engineer has expanded far beyond any original meanings. The same pedantry could apply to the word "developer", too. It's really a weird sort of gatekeeping of a word.