4 ms·
Prompts are Programs
- 1oooqooq 2y agomore like undocumented api requests to a remote dev box
- llm_trw 2y agoNot if you have the weights of the model locally.
- deleted 2y ago[deleted]
- esperent 2y agoAre there any other programming languages that are inherently as imprecise as prompting an LLM? From my knowledge, there isn't, and that seems to be a fundamental diff between prompts and code.
- skirge 2y ago"Take EXPLOR, for example: here is a programming language with probabilistic execution (every instruction specifies the probability with which it should be executed) and no variables -- every instruction operates on either the screen image or other instructions" https://esolangs.org/wiki/Prehistory_of_esoteric_programming_languages https://esolangs.org/wiki/Prehistory_of_esoteric_programming...
- esperent 2y agoThanks for sharing, that was an interesting read. > every instruction specifies the probability with which it should be executed Is this still precise, in a way that prompting llms isn't? I'd need to explore the language in more depth to answer that, but if it does what I assume which is basically a string of instructions like do x with probability y, then it's still precise, except we're using probability theory in addition to/instead of standard programming logic.
- gus_massa 2y agoIt has been tried and failed multiple times. I never used one, but IIRC the problem is that you must guess what the language will parse and what it will do. For example, if you use c in Wolfram Alpha, sometimes it is interpreted as the speed of light, sometimes as centigrade, sometimes as a constant and sometimes as an error. And sometimes minor changes in other part of the question changes the interpretation of c.
- xnx 2y agoThis seems to have similar ideas to an post from a year ago by François Chollet: https://fchollet.substack.com/p/how-i-think-about-llm-prompt-engineering https://fchollet.substack.com/p/how-i-think-about-llm-prompt...
- tgv 2y agoCommand lines are programs. Keystrokes on a calculator are programs. Cooking recipes are programs. It’s deliberate antagonistic widening of a word’s imprecise meaning, possibly to get some notoriety. Academic clickbait.
- esperent 2y agoWell, I would argue that cooking recipes are algorithms, at least. In fact, they were probably the very first example of an algorithm that was given to you in your first Intro to Algorithms class.
- tgv 2y agoBut not programs. Not in the sense of SIGPLAN.
- ben_w 2y agoI'm sure I've seen "cooking recipes are programs" back in the late 80s…
- sebtron 2y ago> We call the direct input to the chatbot the user prompt. Another, more complex prompt is the prompt that was written to process the user prompt, which is often called the system prompt. I thought this was called a query, and a prompt was text / symbols written by the system to prompt the user to enter a command or query. Is this meaning only used by AI folks?
- deleted 2y ago[deleted]
- petters 2y agoNo, one meaning is "an act of encouraging a hesitating speaker," which seems OK to use in this context
- MobiusHorizons 2y agoI believe the use comes from the concept of a writing prompt, like one might be given for writing an essay or creative writing assignment.
- deleted 2y ago[deleted]
- rgbswan 2y agoexcept English or any natural language leaves too much wiggle room and LLMs have .... 0 mappings to machine code. Soon LLMs will do Feature Bloat, Surveillance and Privacy Breaches as a service - no prompt necessary, thank us very much - and then the kids won't even notice who slipped what into their metaphorical drink from that digital fountain. It won't be obvious anymore. Thanks to people saying things like "AI" and "prompts are programs".
- mplewis 2y agoNo, they’re bullshit. Stop selling bullshit.
- netcan 2y agoAny interesting examples of methods to make LLMs more "programmable?"
- tilt_error 2y agoDSPy?
- cpfiffer 2y agoI work at .txt, where we work on exactly this. We use structured generation to make programmable language model systems. Here's two examples I worte + love. One makes a (sort of) internally consistent infinite world generator with RAG + Outlines: https://github.com/dottxt-ai/demos/tree/main/lore-generator https://github.com/dottxt-ai/demos/tree/main/lore-generator This one makes two characters up and then builds a simple combat loop where they fight each other: https://github.com/dottxt-ai/demos/tree/main/nouscon-24 https://github.com/dottxt-ai/demos/tree/main/nouscon-24 Both are simple examples of basic programming with language model outputs.
- jl6 2y agovon Neumann taught us that code and data are the same thing and the distinction is a matter of context and perspective. Some bits are code-like and some are data-like, and some inhabit a gray area where no bright lines are to be found. It’s provocative to claim that prompts are more like code than like data. Feels to me more like they sit in that gray area alongside things like HTML, config files, regular expressions, and spreadsheets. In practical terms the more pressing question is what kind of tools are needed to manage prompt bits. Is it beneficial to manage prompts using traditional software engineering tooling? This post says yes. Maybe, but I can see traditional data management tooling being equally applicable.
- ithkuil 2y agoCode Is data that expresses instructions to some processing unit for it to do tasks. Prompts fit that description very well so I don't find that provocative or controversial at all. The fact that traditionally the execution unit was very rigid and that the language used to express instructions was distant from human natural language is more of a consequence of an implementation rather than a fundamental distinction between code and data
- NobleLie 2y ago> expresses instructions To instruct is akin to providing an input. Where do we input to? The program. The program is programmed; but most importantly, the input is not programmed in the sense the program is programmed. It is, perhaps, thoughfully designed (or not). This role is best described as Prompt Designing or Prompt Engineering, not Prompt "Programming". In any case, it is indeed very important. But in my respectful opinion, it is not programming. It's kind of like thinking that entering a query into a search engine is programming - a host of techniques can be utilized to optimize how that input effects the output (tomes can be written on this just as they can for LLM prompt design by now). I can kind of defend that premise but it breaks the meaning of conventional usages, and creates more confusion than it brings clarity.
- ithkuil 2y ago
- diwank 2y agoCompletely agree. We recently published similar thoughts: https://8factoragent.org/ https://8factoragent.org/
- NobleLie 2y agoNo...prompts are definitely not programs, prompts are inputs to programs. How someone can write a blog post on such a demonstrably fallacious premise is beyond me. I am beginning to write a blog post in rebuttal, because it's so fascinating.
- tiborsaas 2y agoSo some text in helloworld.c is no a program because it's passed to another program as an input in order to be executed? I'd consider it a program, but surely not a deterministic one.
- deleted 2y ago[deleted]
- jgalt212 2y agoIf prompts are programs (which I don't think they are, but they are closer program than search queries), then LLMs don't work as advertised. Which, of course, they don't if you've ever seen a mind-blowing demo and then tried to run some of your own tasks.