5 ms·

Famous quote: "Wouldn't it be nice if our machines were smart enough to allow programming in natural language?". Well, natural languages are most suitable for

• john2x 4y ago
  We need a DSL that converts/compiles into natural language for the AI.
• westoncb 4y ago
  In (language model-backed) natural language interfaces the loss of precision can be made up via iteration. If it were about getting it right on the first try this would be a dead end but there's no need for that restriction.
  • roflyear 4y ago
    Yeah maybe! Just like if you have enough monkeys iterating on enough keyboards...
    • westoncb 4y ago
      Just a few monkeys should be fine ;) If this weren't the case then it wouldn't be possible for (e.g.) the software industry to exist as it does: non-technical folks using natural language are able to converse with engineers who take informal descriptions and turn them into code, often leaning heavily on iteration the bring code and spec into conformance.
      • roflyear 4y ago
        "should be fine" - how do you determine this? There have been many, many cases where I was not able to get GPT-4 to "understand" my problem. No matter how much I tried (until I hit the rate limit for those hours, anyway). People are throwing these absolutes around, and it's just not totally true. Much of an engineer's job is to try and implement the correct solution for imperfect requirements, then to go back and quickly fix things to match the real requirements.
  • btown 4y ago
    And specifically, if the agent implementing the requested actions has a mental model of the domain that is similar to the mental model held by the imprecise specification writer, then the specification doesn't need to be precise to have a high probability of being implemented right the first time! The miracle here is that LLMs don't even need fine-tuning to get to that level - and that's unprecedented for non-human agents.
  • ZephyrBlu 4y ago
    Legibility is generally a requirement though. At least is currently is.
  • SanderNL 4y ago
    What makes us think that encoding all functionality using natural language is somehow more compact than using code? Sure, describing a website with a button is easy, but I don’t quite see how you would describe Kubernetes or a device driver without writing a book in “legalese”. Or try formulating a math proof with natural language.
    • flangola7 4y ago
      Who said you need to need to describe Kubernetes? "Make my programs work at any scale". Boom
      • SanderNL 4y ago
        That just.. doesn’t work that way. Edit: besides, if it could work you lose the competitive edge. I could describe a much faster more cost effective system which the machine can implement. And we are off to the races again..
        • flangola7 4y ago
          If/when it is a generally capable intelligence it very much does work that way. That's how it worked with humans. We saw that stuff needed to work at scale and created something to achieve that.
          • SanderNL 4y ago
            I guess you are right about AGI. I think (hope) that is still off in the distance. We’d have other problems than scaling crud apps, I think.
            • flangola7 4y ago
              >I think (hope) that is still off in the distance. It's being rumored that OpenAI is currently training GPT-5 which will be ready in December, and that many people in the company think it will be a human or better level AGI. Even if it isn't the consistently supersonic jumps they are making every generation suggests we don't have long until human brains are outmoded legacy hardware. >We’d have other problems than scaling crud apps, I think. Ever since I first interacted with the original GPT-3 in 2020 I've had the realization that our future was going to be curtailed and distorted into an inconceivable Escher piece. It seems that future is nearly upon us.
• jxf 4y ago
  While I agree, I don't think this is what Dijkstra was speaking about. A lot of the "AI interfaces" are just underpinning the real technical representation. For example, if you say "Which customers recently bought product #721?" this might generate a SQL query like "select * from customers where ...". The interface is not the ChatGPT text box; it's SQL. The ChatGPT text box is just an assistant to help you do the correct thing (or at least, that's the way it should be used).
  • lwhi 4y ago
    Your example isn't very comparible to tasks involved in creating, texturing, lighting, animating and rendering 3D models using spline and mesh topologies.
    • jxf 4y ago
      I would say it's applicable to anything with a programming language or DSL for it, which this does.
    • ibejoeb 4y ago
      All of that seems very plausible, right? "Make a juvenile elephant." "Make his ears comically large." "Bigger." "Give him a little floppy hat." I don't think it's out of the question for these kinds of commands to result in the correct outcomes. Now, maybe I can adjust the ear size more precisely with my mouse, but it probably saved me a bunch of work.
• whateveracct 4y ago
  I really feel this. Whenever I try to use ChatGPT, I feel both slow and strained. My usual workflow involves me thinking very few English sentences in my head - I move faster thanks to abstract thought, and I don't think I'm alone.
  • whitemary 4y ago
    Same. Somewhat ironically, my art school vocabulary has come in very handy when writing art generator prompts.
• TheDong 4y ago
  On the other hand, we already specify programs in natural language, and in this case chatGPT is really taking a "specification language" as input. When a client wants a button on a webpage, they don't send the web designer a legaleze document describing the dimensions of the button. They usually don't even tell the designer what font to use. The web designer pattern matches the client's english request to the dozens of websites they've built, similar buttons they've seen and used, and then either asks for clarification, or specifies it clearly to the machine in a more specific way. Is that different from the chatGPT flow? Honestly, we also already mostly use english for programming too, not just design. Most of programming now is glueing together libraries, and libraries don't provide a formal logical specification of how each function works. No, they provide english documentation saying something like "http.get(url) returns an httpresponse object or an error". That's far from an actual mathematical specification of how it works, but the plain english definition is enough that most programmers won't ever look at the implementation, the actually correct specification, because the english docs are fine.
  • viraptor 4y ago
    > Is that different from the chatGPT flow? The designer knows the context of the question, the website, the previous meetings about the design styles, possibly information about the visitor demographics and goals, knows the implicit rules about approvals and company hierarchy, knows the toolset used, the project conventions, the previous issues, the test procedures, etc. The equivalent of telling a designer where you want a new button would be equivalent to feeding a small book of the implicit context into ChatGPT and without access to visual feedback you could still end up with an off-screen button that passes all the tests and doesn't do anything. The "fun" part is that for simple tasks 90% of the time it will work every time - then it will do something completely stupid. > they don't send the web designer a legaleze That's the implicit context. (And yeah, bad assumptions about what both sides agree on causes problems for people too) Also, chatgpt will try to make you happy. You want a green button here? You'll get a green button here. A designer instead will tell you it's a terrible idea and breaks accessibility.
• einhverfr 4y ago
  So, really, all we need then is a language sufficiently precise enough to specify what a program needs to do and we can then feed it into a program which can write software that implements that specification, possibly adopting safe transformations of it into equivalent forms. Now, that safely describes a modern, optimizing C compiler.....
• courseofaction 4y ago
  fantastic quote, thank you
• georgelyon 4y ago
  I’ve brought this up in another thread, but we already have evidence that in certain cases it is desirable to trade off specificity for just “getting something done”, that evidence is Python. Python succeeds despite not having a serious type system, because often times you can get something that achieves your goal despite the 25 bugs in your program that would have been caught by a real type system. On the other side of the coin, there’s C++, which is usually doing the heavy lifting underneath the underspecified-but-sufficient Python code. My guess is that as LLMs evolve, they will more naturally fill this niche and you will have high-level, underspecified “code” (prompts) that then glued together more formal libraries (like OpenAI’s plugins).
  • ehnto 4y ago
    I think there exists room for an intermediate layer, a framework for AI to write code in specifically. With larger "target" code blocks it can stitch together. Code is designed around humans to allow them to write specific, abritrary and precise logic with a lot of minuate, but LLMs might be better off thinking in larger business logic blocks that are testable and "sanctioned" by the framework makers for the specific problem domain.
  • Pxtl 4y ago
    Not only that, but one of the big successes of chat AIs is their conversation. When a program does something wrong, it's hard to figure out what you said wrong. When a char AI misunderstood you, it's often quite easy to explain where the misunderstanding happened and the AI will correct itself.
  • staplerr 4y ago
    Just look at JavaScript. Types don't matter so much, look at the modern web.
    • baq 4y ago
      I’m working with a large js code base right now and can’t get it ported to typescript fast enough.
• thomastjeffery 4y ago
  Yes and no. You've provided an excellent introduction to the problem space, but I think natural language has a larger role in formalisms than you might expect. The most familiar formal language grammars to most people here are programming languages. The difference between them and natural language has been categorized as the difference between "context-free grammar" and "context-dependent grammar". The most popular context-free language is mathematics. The language of math provides an excellent grammar for expressing logical relationships. You can take an equation, write it in math, and transform it into a different equivalent representation. But why? Arithmetic. The Pythagorean Theorem would be wholly inconsequential if we didn't have an interest in calculating triangles. The application of math exists outside the grammar itself. This is why you, and everyone else here, grew up with story problems in math class. Similarly, programming languages provide excellent utility for describing explicit computational behavior. What they are missing is the reason why that behavior should exist at all. Programs are surrounded by moats of incompatible context: it takes explicit design to coordinate them together. If we can be explicit about the context in which a formalism exists, we could eliminate the need for ambiguity. With that work done, the incompatibility between software could be factored out. We could be precise about what we mean, and clear about what we infer. We could factor out all semantic arguments, and all logically fallacious positions. We could make empathy itself into software. That is the dream of Natural Language Processing. I think that dream is achievable, but certainly not through implicit text models (LLMs). We need an explicit symbolic approach like parsing. If you're interested, I have been chewing on an idea that might work.
  • xvector 4y ago
    > Similarly, programming languages provide excellent utility for describing explicit computational behavior. What they are missing is the reason why that behavior should exist at all. Programs are surrounded by moats of incompatible context: it takes explicit design to coordinate them together. Such a well written reply! This puts into words a lot of my thoughts around programming today and how NLP can help.
• keithalewis 4y ago
  Visual Basic is so easy secretaries will be able to write programs. Sometimes smart people say dumb things and it takes a while to figure out they are wrong.
• sillysaurusx 4y ago
  Meanwhile, Star Trek has been using natural language for a few decades now, and it works pretty well. Considering they also predicted iPads, we might want to take it at face value.
  • swyx 4y ago
    star trek is also fictional where things happen as the writers wish it to… the real world is not subject to such restrictions
    • sillysaurusx 4y ago
      Then why do our iPads seem to match their PADDs almost exactly?
      • marginalia_nu 4y ago
        Because Apple engineers watch Star Trek too, more than likely. Also, the wax tablets of antiquity also strongly resemble iPads. It's a fairly old invention. Might as well argue the ancient Greeks invented iPads.
• hiAndrewQuinn 4y ago
  "Wouldn't it be nice?" "But it isn't so." An is-versus-ought style mistake.
• 6gvONxR4sf7o 4y ago
  If you want something unambiguous in a single pass, sure. But interaction with some back and forth can dig into the ambiguities that matter quickly, ignoring those that don’t. Dialog is a whole separate model. So maybe natural language is a terrible specification language but a great interaction language.
• ricochet11 4y ago
  Similar vibes: "The Sketchpad system makes it possible for a man and a computer to converse rapidly through the medium of line drawings. Heretofore, most interaction between men and computers has been slowed down by the need to reduce all communication to written statements that can be typed" - Sutherland
• Llamamoe 4y ago
  Natural language doesn't NEED to be precise- that's what theory of mind is for, to allow efficient inference of meaning from objectively-ambiguous communication. It's not perfect, but LLMs appear capable of making the same kind of inference, sufficiently that it'll inevitably be possible to program in natural language sooner or later with minimal or no manual double-checking