4 ms·

This seems misguided. Or at least, I can't see the value of it. It just seems like a classic JS community take - shape a thing so it fits with the familiar ecos

This seems misguided. Or at least, I can't see the value of it. It just seems like a classic JS community take - shape a thing so it fits with the familiar ecosystem, constrain it with structure and types and such, and keep grinding. I think the author misses the point by a million miles: part of the attractiveness of LLMs is that you don't have to bother with a formally structured language to achieve results. GPT effortlessly bridges the gap between programming and human languages - making some structured prompt language seems.. pointless. Almost like an artifact of a mind that finds comfort in constraint, at a time when we're given freedom to roam.

inductive_magic 4y ago
Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. The second you start incorporating LLMs in your product chain, you need the abilities to a) efficiently instruct them without wasting tokens and b) interpret their outputs according to the structure required downstream. When you're just playing, sure, you can talk to it like a toddler would, not like a lawyer/mathematician/programmer. When you actually want to create the kind of software that GPT suddenly enables, you need to come up with an interface between it and, well, other software. I'm not saying SudoLang is the adequate solution to that, but I wouldn't say that it "misses the point by a million miles". That's quite the statement there.
- thisoneworks 4y ago
  Instead of relying on one big model (and all it's flaws), aren't you better off having separate smaller models? Better for auditing
- deleted 4y ago
  [deleted]
- groby_b 4y ago
  "Language without formalization is unreliable; " What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec. If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)
  - ericelliott 4y ago
    Unit tests. SudoLang has a port of the Riteway unit testing framework, and programs can be stepped through in debug mode to show intermediate processes. Such debugging sessions could be used to fine train future models on algorithmic thinking. This stuff is experimental and not 100% yet, but it does mostly work today, just days after SudoLang was specified. And you don't even need to paste the spec: the whole language is inferable by GPT-4.
- brushfoot 4y ago
  > Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. That's the purpose of plugins/LangChain. For deterministic tasks, don't use an LLM. Whatever you use for process orchestration should be able to make use of the LLM where it's a good fit and something else where it's not. > you need the abilities to a) efficiently instruct them without wasting tokens Plain old brevity. There's no need for semicolons and parentheses and the level of punctiliousness that a formal language imposes. > b) interpret their outputs according to the structure required downstream. You can include downstream structure in your prompt. Still no need to design a language for it. LLMs understand imperatives like "structure your output like this."
- dustingetz 4y ago
  You're both right a picture speaks a thousand words; a formula speaks a thousand pictures Electromagnetism is described with formulas. And yet, once you have the formulas, we use natural language to give meaning to the constituent semantic elements in the context of our reality. Plug the appliance into the 120V outlet. How much of what we do is just organizing pre-built components? Coding has a technical debt problem, but that's caused by human conflict of interests / principle agent problem. We'll have the Maxwell's Equations of CRUD apps soon enough and the Tesla/Edisons with the applications will follow shortly after that.

inductive_magic 4y ago

Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. The second you start incorporating LLMs in your product chain, you need the abilities to a) efficiently instruct them without wasting tokens and b) interpret their outputs according to the structure required downstream. When you're just playing, sure, you can talk to it like a toddler would, not like a lawyer/mathematician/programmer. When you actually want to create the kind of software that GPT suddenly enables, you need to come up with an interface between it and, well, other software. I'm not saying SudoLang is the adequate solution to that, but I wouldn't say that it "misses the point by a million miles". That's quite the statement there.

thisoneworks 4y ago
Instead of relying on one big model (and all it's flaws), aren't you better off having separate smaller models? Better for auditing
deleted 4y ago
[deleted]
groby_b 4y ago
"Language without formalization is unreliable; " What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec. If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)
- ericelliott 4y ago
  Unit tests. SudoLang has a port of the Riteway unit testing framework, and programs can be stepped through in debug mode to show intermediate processes. Such debugging sessions could be used to fine train future models on algorithmic thinking. This stuff is experimental and not 100% yet, but it does mostly work today, just days after SudoLang was specified. And you don't even need to paste the spec: the whole language is inferable by GPT-4.
brushfoot 4y ago
> Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. That's the purpose of plugins/LangChain. For deterministic tasks, don't use an LLM. Whatever you use for process orchestration should be able to make use of the LLM where it's a good fit and something else where it's not. > you need the abilities to a) efficiently instruct them without wasting tokens Plain old brevity. There's no need for semicolons and parentheses and the level of punctiliousness that a formal language imposes. > b) interpret their outputs according to the structure required downstream. You can include downstream structure in your prompt. Still no need to design a language for it. LLMs understand imperatives like "structure your output like this."
dustingetz 4y ago
You're both right a picture speaks a thousand words; a formula speaks a thousand pictures Electromagnetism is described with formulas. And yet, once you have the formulas, we use natural language to give meaning to the constituent semantic elements in the context of our reality. Plug the appliance into the 120V outlet. How much of what we do is just organizing pre-built components? Coding has a technical debt problem, but that's caused by human conflict of interests / principle agent problem. We'll have the Maxwell's Equations of CRUD apps soon enough and the Tesla/Edisons with the applications will follow shortly after that.