4 ms·

Natural language isn’t made to be precise that’s why we use a subset in programming languages. So you either need lots of extra text to remove the ambiguity of

by croes 2y ago

Natural language isn’t made to be precise that’s why we use a subset in programming languages. So you either need lots of extra text to remove the ambiguity of natural language if you use AI or you need a special precise subset to communicate with AI and that’s just programming with extra steps.

oxfordmale 2y ago
Yes, let's devise a more precise way to give AI instructions. Let's call it pAIthon. This will allow powers that be, like Zuckerberg to save face and claim that AI has replaced mid-level developers and enable developers to rebrand themselves as pAIthon programmers. Joking aside, this is likely where we will end up, just with a slightly higher programming interface, making developers more productive.
- dylan604 2y ago
  man, pAIthon was just sitting right there for the taking
  - deleted 2y ago
    [deleted]
  - oxfordmale 2y ago
    Thanks for pointing it out :-)
empath75 2y ago
AIs actually are very good at this. They wouldn't be able to write code at all otherwise. If you're careful in your prompting, they'll make fewer assumptions and ask clarifying questions before going ahead and writing code.
- 9rx 2y ago
  > If you're careful in your prompting In other words, if you replace natural language with a programming language then the computer will do a good job of interpreting your intent. But that's always been true, so...
  - benatkin 2y ago
    Being careful in your prompting doesn’t imply that. That can also be thought of as just using natural language well.
    - 9rx 2y ago
      What separates natural language from programming language is that natural language doesn't have to be careful. Once you have to be careful, you are programming.
      - benatkin 2y ago
        It does have to be careful at times if you’re going to be effective with natural language.
        9rx 2y ago
        Certainly there is a need for care outside of computers too, like in law, but legal documents are a prime example of programs. That's programming, written using a programming language, not natural language. It is decidedly not the same language you would use for casual conversation and generally requires technical expertise to understand.
        benatkin 2y ago
        People can often be observed to be deliberately making an effort in casual, social, natural language conversation. It flows for some people more than others. Try watching Big Bang Theory and see characters at times being deliberate with their words and at other times responding automatically. An LLM can do increasingly well as a fly on the wall, but it’s common for people using an LLM to be less collaborative with an LLM and for them to expect the LLM to structure the conversation. Hence the suggestion to be careful in your prompting.
- oxfordmale 2y ago
  AI is very good at this. Unfortunately, humans tend to be super bad at providing detailed verbal instructions.
  - indymike 2y ago
    Languages used for day to day communication between humans do not have the specificity needed for detailed instructions... even to other humans. We out of band context (body language, social norms, tradition, knowledge of a person) quite a bit more than you would think.
    - nomel 2y ago
      Programming languages, which are human language, are purpose built for this. Anyone working in the domain of precise specifications uses them, or something very similar (for example, engineering, writing contracts, etc), often daily. ;) They all usually build down to a subset of english, because near caveman speak is enough to define things with precision.
  - nomel 2y ago
    Then those same humans won't be able to reason about code, or the problem spaces they're working in, regardless, since it's all fundamentally about precise specifics.
- LordDragonfang 2y ago
  > they'll make fewer assumptions and ask clarifying questions before going ahead and writing code. Which model are you talking about here? Because with ChatGPT, I struggle with getting it to ask any clarifying questions before just dumping code filled with placeholders I don't want, even when I explicitly prompt it to ask for clarification.
- croes 2y ago
  AI is a little bit like Occam‘s razor, when you say hoofbeats, you get horses. Bad if you need Zebras.
- foobiekr 2y ago
  I don’t think I’ve ever seen an llm in any context ask for clarification. Is that a real thing?
deleted 2y ago
[deleted]
Klaster_1 2y ago
A lot of extra text usually means prior requirements, meeting transcripts, screen share recordings, chat history, Jira tickets and so on - the same information developers use to produce a result that satisfies the stakeholders and does the job. This seems like a straightforward direction solvable with more compute and more efficient memory. I think this will be the way it pans outs. Real projects don't require an infinitely detailed specification either, you usually stop where it no longer meaningfully moves you towards the goal. The whole premise of AI developer automation, IMO, is that if a human can develop a thing, then AI should be able too, given the same input.
- cube2222 2y ago
  We are kind of actually there already. With a 200k token window like Claude has you can already dump a lot of design docs / transcripts / etc. at it.
  - rightisleft 2y ago
    Its all about the context window. Even the new Mistral Codestral-2501 256K CW does a great job. If you use cline with any large context model the results can be pretty amazing. It's not close to self guiding, You still need to break down and analyze the problem and provide clear and relevant instructions. IE you need to be a great architect. Once you are stable on the direction, its awe inspiring to watch it do the bulk if the implementation. I do agree that there is space to improve over embedded chat windows in IDEs. Solutions will come in time.
    - selectodude 2y ago
      Issue I have with Cline that I don't run into with, say, Aider, is that I find Cline to be like 10x more expensive. The number of tokens it blows through is incredible. Is that just me?
  - mollyporph 2y ago
    And Gemini has 2m token window. Which is about 10 minutes of video for example.
65 2y ago
We're going to create SQL all over again, aren't we?
- lelanthran 2y ago
  A more modern COBOL maybe.
  - 9rx 2y ago
    So SQL?
spacemanspiff01 2y ago
Or a proposal/feedback process. Ala you are hired by non technical person to build something, you generate requirements and a proposed solution. You then propose that solution, they give feedback. Having a feedback loop is the only way viable for this. Sure, the client could give you a book on what they want, but often people do not know their edge cases, what issues may arise/etc.
pjc50 2y ago
There was a wave of this previously in programming: https://en.wikipedia.org/wiki/The_Last_One_(software) https://en.wikipedia.org/wiki/The_Last_One_(software) All the same buzzwords, including "AI"! In 1981!
dylan604 2y ago
> and that’s just programming with extra steps. If you know how to program, then I agree and part of why I don't see the point. If you don't know how to program, than the prompt isn't much different than providing the specs/requirements to a programmer.
kokanee 2y ago
> or you need a special precise subset to communicate with AI haha, I just imagined sending TypeScript to ChatGPT and having it spit my TypeScript back to me. "See guys, if you just use Turing-complete logically unambiguous input, you get perfect output!"
- charlieyu1 2y ago
  I guess we could have LLM to translate natural language to some precise subset, get it processed, then translate the output back to natural language
thomastjeffery 2y ago
Natural language can be precise, but only in context. The struggle is to provide a context that disambiguates the way you want it to. LLMs solve this problem by avoiding it entirely: they stay ambiguous, and just give you the most familiar context, letting you change direction with more prompts. It's a cool approach, but it's often not worth the extra steps, and sometimes your context window can't fit enough steps anyway. My big idea (the Story Empathizer) is to restructure this interaction such that the only work left to the user is to decide which context suits their purpose best. Given enough context instances (I call them backstories), this approach to natural language processing could recursively eliminate much of its own ambiguity, leaving very little work for us to do in the end. Right now my biggest struggle is figuring out what the foundational backstories will be, and writing them.
- skydhash 2y ago
  That’s what programming languages are: You define a context, then you see that you can shorten the notation to symbol character: Like “The symbol a will refer to the value of type string and content ‘abcd’ and cannot refer to anything else for its life time” get you: const a = “abcd” That is called semantics. Programming is mostly fitting the vagueness inherent to natural languages to the precise context of the programming language.
  - thomastjeffery 2y ago
    Yes, but programming languages are categorically limited to context-free grammar. This means that every expression written in a programming language is explicitly defined to have precisely one meaning. The advantage of natural language is that we can write ambiguously defined expressions, and infer their meaning arbitrarily with context. This means that we can write with fewer unique expressions. It also means that context itself can be more directly involved in the content of what we write. In context-free grammar, we can only express "what" and "how"; never "why". Instead, the "why" is encoded into every decision of the design and implementation of what we are writing. If we could leverage ambiguous language, then we could factor out the "why", and implement it later using context.

by croes 2y ago

oxfordmale 2y ago

Yes, let's devise a more precise way to give AI instructions. Let's call it pAIthon. This will allow powers that be, like Zuckerberg to save face and claim that AI has replaced mid-level developers and enable developers to rebrand themselves as pAIthon programmers. Joking aside, this is likely where we will end up, just with a slightly higher programming interface, making developers more productive.

dylan604 2y ago
man, pAIthon was just sitting right there for the taking
- deleted 2y ago
  [deleted]
- oxfordmale 2y ago
  Thanks for pointing it out :-)

empath75 2y ago

AIs actually are very good at this. They wouldn't be able to write code at all otherwise. If you're careful in your prompting, they'll make fewer assumptions and ask clarifying questions before going ahead and writing code.

9rx 2y ago
> If you're careful in your prompting In other words, if you replace natural language with a programming language then the computer will do a good job of interpreting your intent. But that's always been true, so...
- benatkin 2y ago
  Being careful in your prompting doesn’t imply that. That can also be thought of as just using natural language well.
  - 9rx 2y ago
    What separates natural language from programming language is that natural language doesn't have to be careful. Once you have to be careful, you are programming.
    - benatkin 2y ago
      It does have to be careful at times if you’re going to be effective with natural language.
      - 9rx 2y ago
        Certainly there is a need for care outside of computers too, like in law, but legal documents are a prime example of programs. That's programming, written using a programming language, not natural language. It is decidedly not the same language you would use for casual conversation and generally requires technical expertise to understand.
        benatkin 2y ago
        People can often be observed to be deliberately making an effort in casual, social, natural language conversation. It flows for some people more than others. Try watching Big Bang Theory and see characters at times being deliberate with their words and at other times responding automatically. An LLM can do increasingly well as a fly on the wall, but it’s common for people using an LLM to be less collaborative with an LLM and for them to expect the LLM to structure the conversation. Hence the suggestion to be careful in your prompting.
oxfordmale 2y ago
AI is very good at this. Unfortunately, humans tend to be super bad at providing detailed verbal instructions.
- indymike 2y ago
  Languages used for day to day communication between humans do not have the specificity needed for detailed instructions... even to other humans. We out of band context (body language, social norms, tradition, knowledge of a person) quite a bit more than you would think.
  - nomel 2y ago
    Programming languages, which are human language, are purpose built for this. Anyone working in the domain of precise specifications uses them, or something very similar (for example, engineering, writing contracts, etc), often daily. ;) They all usually build down to a subset of english, because near caveman speak is enough to define things with precision.
- nomel 2y ago
  Then those same humans won't be able to reason about code, or the problem spaces they're working in, regardless, since it's all fundamentally about precise specifics.
LordDragonfang 2y ago
> they'll make fewer assumptions and ask clarifying questions before going ahead and writing code. Which model are you talking about here? Because with ChatGPT, I struggle with getting it to ask any clarifying questions before just dumping code filled with placeholders I don't want, even when I explicitly prompt it to ask for clarification.
croes 2y ago
AI is a little bit like Occam‘s razor, when you say hoofbeats, you get horses. Bad if you need Zebras.
foobiekr 2y ago
I don’t think I’ve ever seen an llm in any context ask for clarification. Is that a real thing?

deleted 2y ago

[deleted]

Klaster_1 2y ago

A lot of extra text usually means prior requirements, meeting transcripts, screen share recordings, chat history, Jira tickets and so on - the same information developers use to produce a result that satisfies the stakeholders and does the job. This seems like a straightforward direction solvable with more compute and more efficient memory. I think this will be the way it pans outs. Real projects don't require an infinitely detailed specification either, you usually stop where it no longer meaningfully moves you towards the goal. The whole premise of AI developer automation, IMO, is that if a human can develop a thing, then AI should be able too, given the same input.

cube2222 2y ago
We are kind of actually there already. With a 200k token window like Claude has you can already dump a lot of design docs / transcripts / etc. at it.
- rightisleft 2y ago
  Its all about the context window. Even the new Mistral Codestral-2501 256K CW does a great job. If you use cline with any large context model the results can be pretty amazing. It's not close to self guiding, You still need to break down and analyze the problem and provide clear and relevant instructions. IE you need to be a great architect. Once you are stable on the direction, its awe inspiring to watch it do the bulk if the implementation. I do agree that there is space to improve over embedded chat windows in IDEs. Solutions will come in time.
  - selectodude 2y ago
    Issue I have with Cline that I don't run into with, say, Aider, is that I find Cline to be like 10x more expensive. The number of tokens it blows through is incredible. Is that just me?
- mollyporph 2y ago
  And Gemini has 2m token window. Which is about 10 minutes of video for example.

65 2y ago

We're going to create SQL all over again, aren't we?

lelanthran 2y ago
A more modern COBOL maybe.
- 9rx 2y ago
  So SQL?

spacemanspiff01 2y ago

Or a proposal/feedback process. Ala you are hired by non technical person to build something, you generate requirements and a proposed solution. You then propose that solution, they give feedback. Having a feedback loop is the only way viable for this. Sure, the client could give you a book on what they want, but often people do not know their edge cases, what issues may arise/etc.

pjc50 2y ago

There was a wave of this previously in programming: https://en.wikipedia.org/wiki/The_Last_One_(software) https://en.wikipedia.org/wiki/The_Last_One_(software) All the same buzzwords, including "AI"! In 1981!

dylan604 2y ago

> and that’s just programming with extra steps. If you know how to program, then I agree and part of why I don't see the point. If you don't know how to program, than the prompt isn't much different than providing the specs/requirements to a programmer.

kokanee 2y ago

> or you need a special precise subset to communicate with AI haha, I just imagined sending TypeScript to ChatGPT and having it spit my TypeScript back to me. "See guys, if you just use Turing-complete logically unambiguous input, you get perfect output!"

charlieyu1 2y ago
I guess we could have LLM to translate natural language to some precise subset, get it processed, then translate the output back to natural language

thomastjeffery 2y ago

Natural language can be precise, but only in context. The struggle is to provide a context that disambiguates the way you want it to. LLMs solve this problem by avoiding it entirely: they stay ambiguous, and just give you the most familiar context, letting you change direction with more prompts. It's a cool approach, but it's often not worth the extra steps, and sometimes your context window can't fit enough steps anyway. My big idea (the Story Empathizer) is to restructure this interaction such that the only work left to the user is to decide which context suits their purpose best. Given enough context instances (I call them backstories), this approach to natural language processing could recursively eliminate much of its own ambiguity, leaving very little work for us to do in the end. Right now my biggest struggle is figuring out what the foundational backstories will be, and writing them.

skydhash 2y ago
That’s what programming languages are: You define a context, then you see that you can shorten the notation to symbol character: Like “The symbol a will refer to the value of type string and content ‘abcd’ and cannot refer to anything else for its life time” get you: const a = “abcd” That is called semantics. Programming is mostly fitting the vagueness inherent to natural languages to the precise context of the programming language.
- thomastjeffery 2y ago
  Yes, but programming languages are categorically limited to context-free grammar. This means that every expression written in a programming language is explicitly defined to have precisely one meaning. The advantage of natural language is that we can write ambiguously defined expressions, and infer their meaning arbitrarily with context. This means that we can write with fewer unique expressions. It also means that context itself can be more directly involved in the content of what we write. In context-free grammar, we can only express "what" and "how"; never "why". Instead, the "why" is encoded into every decision of the design and implementation of what we are writing. If we could leverage ambiguous language, then we could factor out the "why", and implement it later using context.