3 ms·

Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. The second you start incorporating LLMs in you

Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. The second you start incorporating LLMs in your product chain, you need the abilities to a) efficiently instruct them without wasting tokens and b) interpret their outputs according to the structure required downstream. When you're just playing, sure, you can talk to it like a toddler would, not like a lawyer/mathematician/programmer. When you actually want to create the kind of software that GPT suddenly enables, you need to come up with an interface between it and, well, other software. I'm not saying SudoLang is the adequate solution to that, but I wouldn't say that it "misses the point by a million miles". That's quite the statement there.

thisoneworks 4y ago
Instead of relying on one big model (and all it's flaws), aren't you better off having separate smaller models? Better for auditing
deleted 4y ago
[deleted]
groby_b 4y ago
"Language without formalization is unreliable; " What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec. If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)
- ericelliott 3y ago
  Unit tests. SudoLang has a port of the Riteway unit testing framework, and programs can be stepped through in debug mode to show intermediate processes. Such debugging sessions could be used to fine train future models on algorithmic thinking. This stuff is experimental and not 100% yet, but it does mostly work today, just days after SudoLang was specified. And you don't even need to paste the spec: the whole language is inferable by GPT-4.
brushfoot 4y ago
> Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. That's the purpose of plugins/LangChain. For deterministic tasks, don't use an LLM. Whatever you use for process orchestration should be able to make use of the LLM where it's a good fit and something else where it's not. > you need the abilities to a) efficiently instruct them without wasting tokens Plain old brevity. There's no need for semicolons and parentheses and the level of punctiliousness that a formal language imposes. > b) interpret their outputs according to the structure required downstream. You can include downstream structure in your prompt. Still no need to design a language for it. LLMs understand imperatives like "structure your output like this."
dustingetz 4y ago
You're both right a picture speaks a thousand words; a formula speaks a thousand pictures Electromagnetism is described with formulas. And yet, once you have the formulas, we use natural language to give meaning to the constituent semantic elements in the context of our reality. Plug the appliance into the 120V outlet. How much of what we do is just organizing pre-built components? Coding has a technical debt problem, but that's caused by human conflict of interests / principle agent problem. We'll have the Maxwell's Equations of CRUD apps soon enough and the Tesla/Edisons with the applications will follow shortly after that.

thisoneworks 4y ago

Instead of relying on one big model (and all it's flaws), aren't you better off having separate smaller models? Better for auditing

deleted 4y ago

[deleted]

groby_b 4y ago

"Language without formalization is unreliable; " What makes you believe LLMs will stick to the rules of your formalized language? It's likely. There are only probabilistic guarantees, which kind of obviates the benefits of a formal spec. If you need formalism, wrap it into a formal language for the formal parts. But assuming that a formal language spec prompt avoids probabilistic outcomes? Yes, that is missing it by a million miles. You're missing the strength of LLMs. (Which is unstructured input->semi structured output)

ericelliott 3y ago
Unit tests. SudoLang has a port of the Riteway unit testing framework, and programs can be stepped through in debug mode to show intermediate processes. Such debugging sessions could be used to fine train future models on algorithmic thinking. This stuff is experimental and not 100% yet, but it does mostly work today, just days after SudoLang was specified. And you don't even need to paste the spec: the whole language is inferable by GPT-4.

brushfoot 4y ago

> Language without formalization is unreliable; in any critical domain, clarity is not optional. Law, maths, code. That's the purpose of plugins/LangChain. For deterministic tasks, don't use an LLM. Whatever you use for process orchestration should be able to make use of the LLM where it's a good fit and something else where it's not. > you need the abilities to a) efficiently instruct them without wasting tokens Plain old brevity. There's no need for semicolons and parentheses and the level of punctiliousness that a formal language imposes. > b) interpret their outputs according to the structure required downstream. You can include downstream structure in your prompt. Still no need to design a language for it. LLMs understand imperatives like "structure your output like this."

dustingetz 4y ago

You're both right a picture speaks a thousand words; a formula speaks a thousand pictures Electromagnetism is described with formulas. And yet, once you have the formulas, we use natural language to give meaning to the constituent semantic elements in the context of our reality. Plug the appliance into the 120V outlet. How much of what we do is just organizing pre-built components? Coding has a technical debt problem, but that's caused by human conflict of interests / principle agent problem. We'll have the Maxwell's Equations of CRUD apps soon enough and the Tesla/Edisons with the applications will follow shortly after that.