24 ms·

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code b

People are sticking up for LLMs here and that's cool. I wonder, what if you did the opposite? Take a project of moderate complexity and convert it from code back to natural language using your favorite LLM. Does it provide you with a reasonable description of the behavior and requirements encoded in the source code without losing enough detail to recreate the program? Do you find the resulting natural language description is easier to reason about? I think there's a reason most of the vibe-coded applications we see people demonstrate are rather simple. There is a level of complexity and precision that is hard to manage. Sure, you can define it in plain english, but is the resulting description extensible, understandable, or more descriptive than a precise language? I think there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping.

Affric 2y ago
Sure but we build (leaky) abstractions, and this is even happens in legal texts. Asking an llm to build a graphical app in assembly from an ISA and a driver for the display would give you nothing. But with a mountain of abstractions then it can probably do it. This is not to defend an LLM more to say I think that by providing the right abstractions (reusable components) then I do think it will get you a lot closer.
- fsloth 2y ago
  Being doing toy-examples of non-trivial complexity. Architecting the code so context is obvious and there are clear breadcrumbs everywhere is the key. And the LLM can do most of this. Prototype-> refactor/cleanup -> more features -> refactor / cleanup add architectural notes. If you know what a well architected piece of code is supposed to look like, and you proceed in steps, LLM gets quite far as long as you are handholding it. So this is usable for non-trivial _familiar_ code where typing it all would be slower than prompting the llm. Maintaining LLM context is the key here imo and stopping it when you see weird stuff. So it requires you act as thr senior partner PR:ing everyhting.
  - cdkmoose 2y ago
    This begs the question, how many of the newer generation of developers/engineers "know what a well architected piece of code is supposed to look like"?
    - deleted 2y ago
      [deleted]
- sciencesama 2y ago
  Llm frameworks !!
drpixie 2y ago
> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, most flight planning apps will convert the code to prose. But professional pilots (and ATC, etc) universally prefer the coded format. Is is compact (one line instead of a whole paragraph), the format well defined (I know exactly where to look for the one piece I need), and it's unambiguous and well defined. Same for maths and coding - once you reach a certain level of expertise, the complexity and redundancy of natural language is a greater cost than benefit. This seems to apply to all fields of expertise.
- sim7c00 2y ago
  you guys are not wrong. explain any semi complez program, you will instantly resort to diagrams, tables, flow charts etc. etc. ofcourse, you can get your LLM to be bit evil in its replies, to help you truly. rather than to spoon feed you an unhealthy diet. i forbid my LLM to send me code and tell it to be harsh to me if i ask stupid things. stupid as in, lazy questions. send me the link to the manual/specs with an RTFM or something i can digest and better my undertanding. send links not mazes of words. now i can feel myself grow again as a programmer. as you said. you need to build expertise, not try to find ways around it. with that expertise you can find _better_ ways. but for this, firstly, you need the expertise.
  - azernik 2y ago
    If you don't mind sharing - what's the specific prompt you use to get this to happen, and which LLM do you use it with?
    - artifaxx 2y ago
      I can share a similar approach I'm finding beneficial. I add "Be direct and brutally honest in your feedback. Identify assumptions and cognitive biases to correct for." (I also add a compendium of cognitive biases and examples to the knowledge I give the LLM.
soulofmischief 2y ago
What you're describing is decontextualization. A sufficiently powerful transformer would theoretically be able recontextualize a sufficiently descriptive natural language specification. Likewise, the same or an equivalently powerful transformer should be able to fully capture the logic of a complicated program. We just don't have sufficient transformers yet. I don't see why a complete description of the program's design philosophy as well as complete descriptions of each system and module and interface wouldn't be enough. We already produce code according to project specification and logically fill in the gaps by using context.
- scribu 2y ago
  “Fill in the gaps by using context” is the hard part. You can’t pre-bake the context into an LLM because it doesn’t exist yet. It gets created through the endless back-and-forth between programmers, designers, users etc.
  - soulofmischief 2y ago
    But the end result should be a fully-specced design document. That might theoretically be recoverable from a complete program given a sufficiently powerful transformer.
    - skydhash 2y ago
      Peter Naur would disagree with you. From "Programming as Theory Building": A very important consequence of the Theory Building View is that program revival, that is reestablishing the theory of a program merely from the documentation, is strictly impossible. Lest this consequence may seem un- reasonable it may be noted that the need for revival of an entirely dead program probably will rarely arise, since it is hardly conceivable that the revival would be assigned to new programmers without at least some knowledge of the theory had by the original team. Even so the The- ory Building View suggests strongly that program revival should only be attempted in exceptional situations and with full awareness that it is at best costly, and may lead to a revived theory that differs from the one originally had by the program authors and so may contain discrep- ancies with the program text. The definition of theory used in the article: a person who has or possesses a theory in this sense knows how to do certain things and in addition can support the actual doing with explanations, justi- fications, and answers to queries, about the activity of concern. And the main point on how this relate to programming: - 1 The programmer having the theory of the program can explain how the solution relates to the affairs of the world that it helps to handle. Such an explanation will have to be concerned with the manner in which the af- fairs of the world, both in their overall characteristics and their details, are, in some sense, mapped into the pro- gram text and into any additional documentation. - 2 The programmer having the theory of the program can explain why each part of the program is what it is, in other words is able to support the actual program text with a justification of some sort. The final basis of the justification is and must always remain the programmer’s direct, intuitive knowledge or estimate. - 3 The programmer having the theory of the program is able to respond constructively to any demand for a modification of the program so as to support the affairs of the world in a new manner. Designing how a modifi- cation is best incorporated into an established program depends on the perception of the similarity of the new demand with the operational facilities already built into the program. The kind of similarity that has to be per- ceived is one between aspects of the world.
fluidcruft 2y ago
I'm not so sure it's about precision rather than working memory. My presumption is people struggle to understand sufficiently large prose versions for the same reason a LLM would struggle working with larger prose versions: people have limited working memory. The time needed to reload info from prose is significant. People reading large text works will start highlighting and taking notes and inventing shorthand forms in their notes. Compact forms and abstractions help reduce demands for working memory and information search. So I'm not sure it's about language precision.
- card_zero 2y ago
  So is more compact better? Does K&R's *d++ = *s++; get a pass now?
  - fluidcruft 2y ago
    That's probably analogous to reading levels. So it would depend on the reading level of the intended audience. I haven't used C in almost a decade and I would have to refresh/confirm the precise orders of operations there. I do at least know that I need to refresh and after I look it up it should be fine until I forget it again. For people fluent in the language unlikely to be a big deal. Conceivably, if there were an equivalent of "8th grade reading level" for C that forbade pointer arithmetic on the left hand side of an assignment (for example) it could be reformatted by an LLM fairly easily. Some for loop expressions would probably be significantly less elegant, though. But that seems better that converting it to English. That might actually make a clever tooltip sort of thing--highlight a snippet of code and ask for a dumbed-down version in a popup or even an English translation to explain it. Would save me hitting the reference. APL is another example of dense languages that (some) people like to work in. I personally have never had the time to learn it though.
    - skydhash 2y ago
      > APL is another example of dense languages that (some) people like to work in. I recently learn an array programming language called Uiua[0] and it was fun to solve problems in it (I used the advent of code's ones). Some tree operation was a bit of a pain, but you can get very concise code. And after a bit, you can recognize the symbols very easily (and the editor support was good in Emacs). [0]: https://www.uiua.org/ https://www.uiua.org/
nsonha 2y ago
isn't that just copilot "explain", one of the earliest copilot capabilities. It's definitely helpful to understand new codebases at a high level > there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping. unfortunately they're not in any kind of formal language either
- cubefox 2y ago
  > > there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping. > unfortunately they're not in any kind of formal language either Most formulas made of fancy LaTeX symbols you find in math papers aren't a formal language either. They usually can't be mechanically translated via some parser to an actual formal language like Python or Lean. You would need an advanced LLM for that. But they (the LaTeX formulas) are still more precise than most natural language. I assume something similar is the case with legalese.
- still_grokking 2y ago
  > isn't that just copilot "explain", one of the earliest copilot capabilities. It's definitely helpful to understand new codebases at a high level In my experience this function is quite useless. It will just repeat the code in plain English. It will not explain it.
  - kfajdsl 2y ago
    Haven’t tried copilot but cursor is pretty good at telling me where things are and explaining the high level architecture of medium-largeish codebases, especially if I already vaguely know what I’m looking for. I use this a lot when I need to change some behavior of an open source project that I’m using but previously haven’t touched.
  - yencabulator 2y ago
    I was actually positively surprised at how well even qwen2.5-coder:7b managed to talk through a file of Rust. I'm still a current-day-LLM-programming skeptic but that direction, code->English, seems a lot safer, since English is ambiguous anyway. For example, it recognized some of the code shapes and gave English names that can be googled easier.
jimmydddd 2y ago
--I think there is a reason why legalese is not plain English This is true. Part of the precision of legalese is that the meanings of some terms have already been more precisely defined by the courts.
- xwiz 2y ago
  This opens an interesting possibility for a purely symbol-based legal code. This would probably improve clarity when it came to legal phrases that overlap common English, and you could avoid ambiguity when it came to language constructs, like in this case[1], where some drivers were losing overtime pay because of a comma in the overtime law. [1] https://cases.justia.com/federal/appellate-courts/ca1/16-1901/16-1901-2017-03-13.pdf?ts=1489437006 https://cases.justia.com/federal/appellate-courts/ca1/16-190...
  - deleted 2y ago
    [deleted]
- dongkyun 2y ago
  Yeah, my theory on this has always been that a lot of programming efficiency gains have been the ability to unambiguously define behavior, which mostly comes from drastically restricting the possible states and inputs a program can achieve. The states and inputs that lawyers have to deal with tend to much more vague and imprecise (which is expected if you're dealing with human behavior and not text or some other encodeable input) and so have to rely on inherently ambiguous phrases like "reasonable" and "without undue delay."
eightysixfour 2y ago
Language can carry tremendous amounts of context. For example: > I want a modern navigation app for driving which lets me select intersections that I never want to be routed through. That sentence is low complexity but encodes a massive amount of information. You are probably thinking of a million implementation details that you need to get from that sentence to an actual working app but the opportunity is there, the possibility is there, that that is enough information to get to a working application that solves my need. And just as importantly, if that is enough to get it built, then “can I get that in cornflower blue instead” is easy and the user can iterate from there.
- fourside 2y ago
  You call it context or information but I call it assumptions. There are a ton assumptions in that sentence that an LLM will need to make in order to take that and turn it into a v1. I’m not sure what resulting app you’d get but if you did get a useful starting point, I’d wager the fact that you chose a variation of an existing type of app helped a lot. That is useful, but I’m not sure this is universally useful.
  - acka 2y ago
    This is why we have system prompts (or prompt libraries if you cannot easily modify the system prompt). They can be used to store common assumptions related to your workflow. In this example, setting the system prompt to something like "You are an experienced Android app developer specialising in apps for phone form factor devices" (replacing Android with iOS if needed) would get you a long way.
  - eightysixfour 2y ago
    > There are a ton assumptions in that sentence that an LLM will need to make in order to take that and turn it into a v1. I think you need to think of the LLM less like a developer and more like an entire development shop. The first step is working with the user to define their goals, then to repeat it back to them in some format, then to turn it into code, and to iterate during the work with feedback. My last product development conversation with Claude included it drawing svgs of the interface and asking me if that is what I meant. This is much like how other professional services providers don’t need you to bring them exact specs, they take your needs and translate it to specifications that producers can use - working with an architect, a product designer, etc. They assume things and then confirm them - sometimes on paper and in words, sometimes by showing you prototypes, sometimes by just building the thing. The near to mid future of work for software engineers is in two areas in my mind: 1. Doing things no one has done before. The hard stuff. That’s a small percentage of most code, a large percentage of value generated. 2. Building systems and constraints that these automated development tools work within.
vonneumannstan 2y ago
I think you can basically make the same argument for programming directly in machine code since programming languages are already abstractions.
VoodooJuJu 2y ago
[dead]
1vuio0pswjnm7 2y ago
"Sure, you can define it in plain english, but is the resulting description extensible, understandable, or more descriptive than a precise language? I think there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping." Is this suggesting the reason for legalese is to make documents more "extensible, understable or descriptive" than if written in plain English. What is this reason that the parent thinks legalese is used that "goes beyond gatekeeping". Plain English can be every bit as precise as legalese. It is also unclear that legalese exists for the purpose of gatekeeping. For example, it may be an artifact that survives based on familiarity and laziness. Law students are taught to write in plain English. https://www.law.columbia.edu/sites/default/files/2021-07/plain_english_handout.pdf https://www.law.columbia.edu/sites/default/files/2021-07/pla... In some situations, e.g., drafting SEC filings, use of plain English is required by law. https://www.law.cornell.edu/cfr/text/17/240.13a-20 https://www.law.cornell.edu/cfr/text/17/240.13a-20
- feoren 2y ago
  > Plain English can be every bit as precise as legalese. If you attempt to make "plain English" as precise as legalese, you will get something that is basically legalese. Legalese does also have some variables, like "Party", "Client", etc. This allows for both precision -- repeating the variable name instead of using pronouns or re-identifying who you're talking about -- and also for reusability: you can copy/paste standard language into a document that defines "Client" differently, similar to a subroutine.
_ea1k 2y ago
I've thought about this quite a bit. I think a tool like that would be really useful. I can imagine asking questions like "I think this big codebase exposes a rest interface for receiving some sort of credit check object. Can you find it and show me a sequence diagram for how it is implemented?" The challenge is that the codebase is likely much larger than what would fit into a single codebase. IMO, the LLM really needs to be taught to consume the project incrementally and build up a sort of "mental model" of it to really make this useful. I suspect that a combination of tool usage and RL could produce an incredibly useful tool for this.
cyanydeez 2y ago
the vibe coding seems a lot like the dream of using UML, but in a distinctly different direction, and how in theory (and occasional practice) you can create a two way street, most often these things are one way conversions and while we all desire some level of two way dependency and continual integration to make certain aspects of coding (documentation, testing) to be up to date, the reality is that the generative code aspect always breaks and you're always going to be left with the raw products of these tools and it's rarely going to be a cycle of code -> tool -> code. And thus the ultimate value beyond the bootstrap is lose. We're still going to have AI tools, but seriously complex applications, the ones we pay money for, arn't going to yield many LLM based curation strategies. There will probably be some great documentation and testing ones, but the architetural-code paradigm isnt going to yield any time soon.

Affric 2y ago

Sure but we build (leaky) abstractions, and this is even happens in legal texts. Asking an llm to build a graphical app in assembly from an ISA and a driver for the display would give you nothing. But with a mountain of abstractions then it can probably do it. This is not to defend an LLM more to say I think that by providing the right abstractions (reusable components) then I do think it will get you a lot closer.

fsloth 2y ago
Being doing toy-examples of non-trivial complexity. Architecting the code so context is obvious and there are clear breadcrumbs everywhere is the key. And the LLM can do most of this. Prototype-> refactor/cleanup -> more features -> refactor / cleanup add architectural notes. If you know what a well architected piece of code is supposed to look like, and you proceed in steps, LLM gets quite far as long as you are handholding it. So this is usable for non-trivial _familiar_ code where typing it all would be slower than prompting the llm. Maintaining LLM context is the key here imo and stopping it when you see weird stuff. So it requires you act as thr senior partner PR:ing everyhting.
- cdkmoose 2y ago
  This begs the question, how many of the newer generation of developers/engineers "know what a well architected piece of code is supposed to look like"?
  - deleted 2y ago
    [deleted]
sciencesama 2y ago
Llm frameworks !!

drpixie 2y ago

> Do you find the resulting natural language description is easier to reason about? An example from an different field - aviation weather forecasts and notices are published in a strongly abbreviated and codified form. For example, the weather at Sydney Australia now is: METAR YSSY 031000Z 08005KT CAVOK 22/13 Q1012 RMK RF00.0/000.0 It's almost universal that new pilots ask "why isn't this in words?". And, indeed, most flight planning apps will convert the code to prose. But professional pilots (and ATC, etc) universally prefer the coded format. Is is compact (one line instead of a whole paragraph), the format well defined (I know exactly where to look for the one piece I need), and it's unambiguous and well defined. Same for maths and coding - once you reach a certain level of expertise, the complexity and redundancy of natural language is a greater cost than benefit. This seems to apply to all fields of expertise.

sim7c00 2y ago
you guys are not wrong. explain any semi complez program, you will instantly resort to diagrams, tables, flow charts etc. etc. ofcourse, you can get your LLM to be bit evil in its replies, to help you truly. rather than to spoon feed you an unhealthy diet. i forbid my LLM to send me code and tell it to be harsh to me if i ask stupid things. stupid as in, lazy questions. send me the link to the manual/specs with an RTFM or something i can digest and better my undertanding. send links not mazes of words. now i can feel myself grow again as a programmer. as you said. you need to build expertise, not try to find ways around it. with that expertise you can find _better_ ways. but for this, firstly, you need the expertise.
- azernik 2y ago
  If you don't mind sharing - what's the specific prompt you use to get this to happen, and which LLM do you use it with?
  - artifaxx 2y ago
    I can share a similar approach I'm finding beneficial. I add "Be direct and brutally honest in your feedback. Identify assumptions and cognitive biases to correct for." (I also add a compendium of cognitive biases and examples to the knowledge I give the LLM.

soulofmischief 2y ago

What you're describing is decontextualization. A sufficiently powerful transformer would theoretically be able recontextualize a sufficiently descriptive natural language specification. Likewise, the same or an equivalently powerful transformer should be able to fully capture the logic of a complicated program. We just don't have sufficient transformers yet. I don't see why a complete description of the program's design philosophy as well as complete descriptions of each system and module and interface wouldn't be enough. We already produce code according to project specification and logically fill in the gaps by using context.

scribu 2y ago
“Fill in the gaps by using context” is the hard part. You can’t pre-bake the context into an LLM because it doesn’t exist yet. It gets created through the endless back-and-forth between programmers, designers, users etc.
- soulofmischief 2y ago
  But the end result should be a fully-specced design document. That might theoretically be recoverable from a complete program given a sufficiently powerful transformer.
  - skydhash 2y ago
    Peter Naur would disagree with you. From "Programming as Theory Building": A very important consequence of the Theory Building View is that program revival, that is reestablishing the theory of a program merely from the documentation, is strictly impossible. Lest this consequence may seem un- reasonable it may be noted that the need for revival of an entirely dead program probably will rarely arise, since it is hardly conceivable that the revival would be assigned to new programmers without at least some knowledge of the theory had by the original team. Even so the The- ory Building View suggests strongly that program revival should only be attempted in exceptional situations and with full awareness that it is at best costly, and may lead to a revived theory that differs from the one originally had by the program authors and so may contain discrep- ancies with the program text. The definition of theory used in the article: a person who has or possesses a theory in this sense knows how to do certain things and in addition can support the actual doing with explanations, justi- fications, and answers to queries, about the activity of concern. And the main point on how this relate to programming: - 1 The programmer having the theory of the program can explain how the solution relates to the affairs of the world that it helps to handle. Such an explanation will have to be concerned with the manner in which the af- fairs of the world, both in their overall characteristics and their details, are, in some sense, mapped into the pro- gram text and into any additional documentation. - 2 The programmer having the theory of the program can explain why each part of the program is what it is, in other words is able to support the actual program text with a justification of some sort. The final basis of the justification is and must always remain the programmer’s direct, intuitive knowledge or estimate. - 3 The programmer having the theory of the program is able to respond constructively to any demand for a modification of the program so as to support the affairs of the world in a new manner. Designing how a modifi- cation is best incorporated into an established program depends on the perception of the similarity of the new demand with the operational facilities already built into the program. The kind of similarity that has to be per- ceived is one between aspects of the world.

fluidcruft 2y ago

I'm not so sure it's about precision rather than working memory. My presumption is people struggle to understand sufficiently large prose versions for the same reason a LLM would struggle working with larger prose versions: people have limited working memory. The time needed to reload info from prose is significant. People reading large text works will start highlighting and taking notes and inventing shorthand forms in their notes. Compact forms and abstractions help reduce demands for working memory and information search. So I'm not sure it's about language precision.

card_zero 2y ago
So is more compact better? Does K&R's *d++ = *s++; get a pass now?
- fluidcruft 2y ago
  That's probably analogous to reading levels. So it would depend on the reading level of the intended audience. I haven't used C in almost a decade and I would have to refresh/confirm the precise orders of operations there. I do at least know that I need to refresh and after I look it up it should be fine until I forget it again. For people fluent in the language unlikely to be a big deal. Conceivably, if there were an equivalent of "8th grade reading level" for C that forbade pointer arithmetic on the left hand side of an assignment (for example) it could be reformatted by an LLM fairly easily. Some for loop expressions would probably be significantly less elegant, though. But that seems better that converting it to English. That might actually make a clever tooltip sort of thing--highlight a snippet of code and ask for a dumbed-down version in a popup or even an English translation to explain it. Would save me hitting the reference. APL is another example of dense languages that (some) people like to work in. I personally have never had the time to learn it though.
  - skydhash 2y ago
    > APL is another example of dense languages that (some) people like to work in. I recently learn an array programming language called Uiua[0] and it was fun to solve problems in it (I used the advent of code's ones). Some tree operation was a bit of a pain, but you can get very concise code. And after a bit, you can recognize the symbols very easily (and the editor support was good in Emacs). [0]: https://www.uiua.org/ https://www.uiua.org/

nsonha 2y ago

isn't that just copilot "explain", one of the earliest copilot capabilities. It's definitely helpful to understand new codebases at a high level > there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping. unfortunately they're not in any kind of formal language either

cubefox 2y ago
> > there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping. > unfortunately they're not in any kind of formal language either Most formulas made of fancy LaTeX symbols you find in math papers aren't a formal language either. They usually can't be mechanically translated via some parser to an actual formal language like Python or Lean. You would need an advanced LLM for that. But they (the LaTeX formulas) are still more precise than most natural language. I assume something similar is the case with legalese.
still_grokking 2y ago
> isn't that just copilot "explain", one of the earliest copilot capabilities. It's definitely helpful to understand new codebases at a high level In my experience this function is quite useless. It will just repeat the code in plain English. It will not explain it.
- kfajdsl 2y ago
  Haven’t tried copilot but cursor is pretty good at telling me where things are and explaining the high level architecture of medium-largeish codebases, especially if I already vaguely know what I’m looking for. I use this a lot when I need to change some behavior of an open source project that I’m using but previously haven’t touched.
- yencabulator 2y ago
  I was actually positively surprised at how well even qwen2.5-coder:7b managed to talk through a file of Rust. I'm still a current-day-LLM-programming skeptic but that direction, code->English, seems a lot safer, since English is ambiguous anyway. For example, it recognized some of the code shapes and gave English names that can be googled easier.

jimmydddd 2y ago

--I think there is a reason why legalese is not plain English This is true. Part of the precision of legalese is that the meanings of some terms have already been more precisely defined by the courts.

xwiz 2y ago
This opens an interesting possibility for a purely symbol-based legal code. This would probably improve clarity when it came to legal phrases that overlap common English, and you could avoid ambiguity when it came to language constructs, like in this case[1], where some drivers were losing overtime pay because of a comma in the overtime law. [1] https://cases.justia.com/federal/appellate-courts/ca1/16-1901/16-1901-2017-03-13.pdf?ts=1489437006 https://cases.justia.com/federal/appellate-courts/ca1/16-190...
- deleted 2y ago
  [deleted]
dongkyun 2y ago
Yeah, my theory on this has always been that a lot of programming efficiency gains have been the ability to unambiguously define behavior, which mostly comes from drastically restricting the possible states and inputs a program can achieve. The states and inputs that lawyers have to deal with tend to much more vague and imprecise (which is expected if you're dealing with human behavior and not text or some other encodeable input) and so have to rely on inherently ambiguous phrases like "reasonable" and "without undue delay."

eightysixfour 2y ago

Language can carry tremendous amounts of context. For example: > I want a modern navigation app for driving which lets me select intersections that I never want to be routed through. That sentence is low complexity but encodes a massive amount of information. You are probably thinking of a million implementation details that you need to get from that sentence to an actual working app but the opportunity is there, the possibility is there, that that is enough information to get to a working application that solves my need. And just as importantly, if that is enough to get it built, then “can I get that in cornflower blue instead” is easy and the user can iterate from there.

fourside 2y ago
You call it context or information but I call it assumptions. There are a ton assumptions in that sentence that an LLM will need to make in order to take that and turn it into a v1. I’m not sure what resulting app you’d get but if you did get a useful starting point, I’d wager the fact that you chose a variation of an existing type of app helped a lot. That is useful, but I’m not sure this is universally useful.
- acka 2y ago
  This is why we have system prompts (or prompt libraries if you cannot easily modify the system prompt). They can be used to store common assumptions related to your workflow. In this example, setting the system prompt to something like "You are an experienced Android app developer specialising in apps for phone form factor devices" (replacing Android with iOS if needed) would get you a long way.
- eightysixfour 2y ago
  > There are a ton assumptions in that sentence that an LLM will need to make in order to take that and turn it into a v1. I think you need to think of the LLM less like a developer and more like an entire development shop. The first step is working with the user to define their goals, then to repeat it back to them in some format, then to turn it into code, and to iterate during the work with feedback. My last product development conversation with Claude included it drawing svgs of the interface and asking me if that is what I meant. This is much like how other professional services providers don’t need you to bring them exact specs, they take your needs and translate it to specifications that producers can use - working with an architect, a product designer, etc. They assume things and then confirm them - sometimes on paper and in words, sometimes by showing you prototypes, sometimes by just building the thing. The near to mid future of work for software engineers is in two areas in my mind: 1. Doing things no one has done before. The hard stuff. That’s a small percentage of most code, a large percentage of value generated. 2. Building systems and constraints that these automated development tools work within.

vonneumannstan 2y ago

I think you can basically make the same argument for programming directly in machine code since programming languages are already abstractions.

VoodooJuJu 2y ago

[dead]

1vuio0pswjnm7 2y ago

"Sure, you can define it in plain english, but is the resulting description extensible, understandable, or more descriptive than a precise language? I think there is a reason why legalese is not plain English, and it goes beyond mere gatekeeping." Is this suggesting the reason for legalese is to make documents more "extensible, understable or descriptive" than if written in plain English. What is this reason that the parent thinks legalese is used that "goes beyond gatekeeping". Plain English can be every bit as precise as legalese. It is also unclear that legalese exists for the purpose of gatekeeping. For example, it may be an artifact that survives based on familiarity and laziness. Law students are taught to write in plain English. https://www.law.columbia.edu/sites/default/files/2021-07/plain_english_handout.pdf https://www.law.columbia.edu/sites/default/files/2021-07/pla... In some situations, e.g., drafting SEC filings, use of plain English is required by law. https://www.law.cornell.edu/cfr/text/17/240.13a-20 https://www.law.cornell.edu/cfr/text/17/240.13a-20

feoren 2y ago
> Plain English can be every bit as precise as legalese. If you attempt to make "plain English" as precise as legalese, you will get something that is basically legalese. Legalese does also have some variables, like "Party", "Client", etc. This allows for both precision -- repeating the variable name instead of using pronouns or re-identifying who you're talking about -- and also for reusability: you can copy/paste standard language into a document that defines "Client" differently, similar to a subroutine.

_ea1k 2y ago

I've thought about this quite a bit. I think a tool like that would be really useful. I can imagine asking questions like "I think this big codebase exposes a rest interface for receiving some sort of credit check object. Can you find it and show me a sequence diagram for how it is implemented?" The challenge is that the codebase is likely much larger than what would fit into a single codebase. IMO, the LLM really needs to be taught to consume the project incrementally and build up a sort of "mental model" of it to really make this useful. I suspect that a combination of tool usage and RL could produce an incredibly useful tool for this.

cyanydeez 2y ago

the vibe coding seems a lot like the dream of using UML, but in a distinctly different direction, and how in theory (and occasional practice) you can create a two way street, most often these things are one way conversions and while we all desire some level of two way dependency and continual integration to make certain aspects of coding (documentation, testing) to be up to date, the reality is that the generative code aspect always breaks and you're always going to be left with the raw products of these tools and it's rarely going to be a cycle of code -> tool -> code. And thus the ultimate value beyond the bootstrap is lose. We're still going to have AI tools, but seriously complex applications, the ones we pay money for, arn't going to yield many LLM based curation strategies. There will probably be some great documentation and testing ones, but the architetural-code paradigm isnt going to yield any time soon.