If we think of LLMs as a nonlinear optimization problem, min F(x) with lots of local optima (valleys), then "prompt engineering" is the act of finding initial p

If we think of LLMs as a nonlinear optimization problem, min F(x) with lots of local optima (valleys), then "prompt engineering" is the act of finding initial point x0 (prompt) such that the optimization lands you a good local optimum. For instance, if your x0 was "what is portfolio theory", chances are you'll get a generic answer that's passable but not very good. If you move your x0 to "if you were henry markowitz, how would you use portfolio theory in your own investment strategies", suddenly you get an answer with more texture and practicability. However I think "prompt engineering," understood literally, is neither engineering nor is it generalizable. In optimization, typically we find x0 through heuristics (rules of thumb) or domain knowledge -- the latter being ultra useful. When optimizing a power plant, if you know where the optimal solution roughly lies from ops experience and can guess at a x0, that helps the optimizer find the actual optimum more easily than throwing darts on a wall, especially when there are a few million variables. But this method of find x0 will not work for another domain. I suppose a pure "prompt engineer" would be akin to a role of a research librarian. In very specialized areas (especially in highly technical areas), research librarians are less effective because they lack the context needed to land really good results. In the article, a "prompt engineer" is really a software tester and customer success person, not really an engineer as such.