4 ms·

By "unpredictability", we mean that AIs will return completely different results if a single word is changed to a close synonym, or an adverb or prepositional p

by o11c 1y ago

By "unpredictability", we mean that AIs will return completely different results if a single word is changed to a close synonym, or an adverb or prepositional phrase is moved to a semantically identical location, etc. Very often this simple change will move you from "get the correct answer 90% of the time" (about the best that AIs can do) to "get the correct answer <10% of the time". Whenever people talk about "prompt engineering", they're referring to randomly changing these kinds of things, in hopes of getting a query pattern where you get meaningful results 90% of the time.

bsenftner 1y ago
What you're describing is specifically the subtle nature of LLMs I'm pointing at; that changing of a single word to a close synonym is meaningful. Why and how they are meaningful gets pushback from the developer community, they somehow do not see this as being a topic, a point of engineering proficiency. It is, but requires an understanding of how LLMs encode and retrieve data. The reason changing one word in a prompt to a close synonym changes the reply is because it is the specific words used in a series that is how information is embedded and recovered by LLMs. The 'in a series' aspect is subtle and important. The same topic is in the LLM multiple times, with different levels of treatment from casual to academic. Each treatment from casual to formal uses different words, similar words, but different and that difference is very meaningful. That difference is how seriously the information is being handled. The use of one term versus another term causes a prompt to index into one treatment of the subject versus another. The more formal the terms used, meaning the synonyms used by experts of that area of knowledge, generate the more accurate replies. While the close synonyms generate replies from outsiders of that knowledge, those not using the same phrases as those with the most expertise, the phrases used by those perhaps trying to understand but do not yet? It is not randomly changing things in one's prompts at all. It's understanding the knowledge space one is prompting within such that the prompts generate accurate replies. This requires knowing the knowledge space one prompts within, so one knows the correct formal terms that unlock accurate replies. Plus, knowing that area, one is in a better position to identify hallucination.
- handfuloflight 1y ago
  Words are power, and specifically, specific words are power.
  - bsenftner 1y ago
    Yes! Why do people get so angry about it? "Oh, you're saying I'm hold it wrong?!" Well, actually, yes, If you speak Pascal to Ruby you get syntax errors, and this is the same basic idea. If you want to talk sports to an LLM and you use shit talking sports language, that's what you'll get back. Obvious, right? Same goes for anything formal, and why is that an insult to people to point that out?
    - handfuloflight 1y ago
      For a subset of these detractors, it's their investment and personal moat building into learning syntax which is now being threatened to be obsoleted by natural language programming. Now people with domain knowledge are able to become developers, whereas previously domain experts relied on syntax writers to translate their requirements into reality. The syntax writers may say: "I do more than write syntax! I think in systems, logic, processes, limits, edge cases, etc." The response to that is: you don't need syntax to do that, yet until now syntax was the barrier to technical expression. So ironically, when they show anger it is a form of hypocrisy: they already know that knowing how to write specific words is power. They're just upset that the specific words that matter have changed.
- noduerme 1y ago
  What you are describing is not natural language programming, it's the use of incantations discovered by accident or by trial and error. It's alchemy, not chemistry. That's what people mean when they say it's not reproducible. It's not reproducible according to any useful logical framework that could be generally applied to other cases. There may be some "power" in knowing magical incantations, but mostly it's going to be a series of parlor tricks, since neither you nor anyone else can explain why one prompt produces an algorithm that spits out value X whilst changing a single word to its synonym produces X*-1, or Q, or 14 rabbits. And if you could, why not just type the algorithm yourself? Higher level programming languages may make choices for coders regarding lower level functionality, but they have syntactic and semantic rules that produce logically consistent results. Claiming that such rules exist for LLMs but are so subtle that only the ultra-enlightened such as yourself can understand them begs the question: If hardly anyone can grasp such subtlety, then who exactly are all these massive models being built for?

by o11c 1y ago

bsenftner 1y ago

What you're describing is specifically the subtle nature of LLMs I'm pointing at; that changing of a single word to a close synonym is meaningful. Why and how they are meaningful gets pushback from the developer community, they somehow do not see this as being a topic, a point of engineering proficiency. It is, but requires an understanding of how LLMs encode and retrieve data. The reason changing one word in a prompt to a close synonym changes the reply is because it is the specific words used in a series that is how information is embedded and recovered by LLMs. The 'in a series' aspect is subtle and important. The same topic is in the LLM multiple times, with different levels of treatment from casual to academic. Each treatment from casual to formal uses different words, similar words, but different and that difference is very meaningful. That difference is how seriously the information is being handled. The use of one term versus another term causes a prompt to index into one treatment of the subject versus another. The more formal the terms used, meaning the synonyms used by experts of that area of knowledge, generate the more accurate replies. While the close synonyms generate replies from outsiders of that knowledge, those not using the same phrases as those with the most expertise, the phrases used by those perhaps trying to understand but do not yet? It is not randomly changing things in one's prompts at all. It's understanding the knowledge space one is prompting within such that the prompts generate accurate replies. This requires knowing the knowledge space one prompts within, so one knows the correct formal terms that unlock accurate replies. Plus, knowing that area, one is in a better position to identify hallucination.

handfuloflight 1y ago
Words are power, and specifically, specific words are power.
- bsenftner 1y ago
  Yes! Why do people get so angry about it? "Oh, you're saying I'm hold it wrong?!" Well, actually, yes, If you speak Pascal to Ruby you get syntax errors, and this is the same basic idea. If you want to talk sports to an LLM and you use shit talking sports language, that's what you'll get back. Obvious, right? Same goes for anything formal, and why is that an insult to people to point that out?
  - handfuloflight 1y ago
    For a subset of these detractors, it's their investment and personal moat building into learning syntax which is now being threatened to be obsoleted by natural language programming. Now people with domain knowledge are able to become developers, whereas previously domain experts relied on syntax writers to translate their requirements into reality. The syntax writers may say: "I do more than write syntax! I think in systems, logic, processes, limits, edge cases, etc." The response to that is: you don't need syntax to do that, yet until now syntax was the barrier to technical expression. So ironically, when they show anger it is a form of hypocrisy: they already know that knowing how to write specific words is power. They're just upset that the specific words that matter have changed.
noduerme 1y ago
What you are describing is not natural language programming, it's the use of incantations discovered by accident or by trial and error. It's alchemy, not chemistry. That's what people mean when they say it's not reproducible. It's not reproducible according to any useful logical framework that could be generally applied to other cases. There may be some "power" in knowing magical incantations, but mostly it's going to be a series of parlor tricks, since neither you nor anyone else can explain why one prompt produces an algorithm that spits out value X whilst changing a single word to its synonym produces X*-1, or Q, or 14 rabbits. And if you could, why not just type the algorithm yourself? Higher level programming languages may make choices for coders regarding lower level functionality, but they have syntactic and semantic rules that produce logically consistent results. Claiming that such rules exist for LLMs but are so subtle that only the ultra-enlightened such as yourself can understand them begs the question: If hardly anyone can grasp such subtlety, then who exactly are all these massive models being built for?