4 ms·

I can count the amount of times in my 20 year career that I've had to look at compiler generated assembly on one finger and I've never looked at the machine co

I can count the amount of times in my 20 year career that I've had to look at compiler generated assembly on one finger and I've never looked at the machine code produced by an assembler (other than when I wrote my own as a toy project) is the same true of LLM usage? absolutely not and it never will be, because it's not an abstraction

elAhmo 1y ago
It is an abstraction. Just because you end up looking at what the prompt looks like “under the hood” in whichever language it produced the output, doesn’t mean every user does. Similar as with assembly, you might have not taken a look at it, but there are people that do and could argue the same thing as you. The lines will be very blurry in the near future.
- card_zero 1y ago
  I can fart in the general direction of the code and that's a kind of abstraction too. It distills my intent down to a simple raspberry noise which could then be interpreted by sufficiently intelligent software.
  - joenot443 1y ago
    Sort of like when people argue about "what is art?", some folks find it clever to come up with the most bizarre of examples to point to, as if the entire concept rests on the existence of its members at the fringe. Personally, I think if your farts are an abstraction that you can derive useful meaning from the mapping, who are we to tell you no?
    - card_zero 1y ago
      So, can derive useful meaning from is the point of contention. (Also: bizarre examples = informative edge cases. Sometimes.)
  - chaps 1y ago
    Could you please expand on this thought? I'm curious where this abstraction's inflection points are.
  - Phemist 1y ago
    The final result of course depending on how many r's there are in the raspberry.
- rsynnott 1y ago
  > Just because you end up looking at what the prompt looks like “under the hood” in whichever language it produced the output, doesn’t mean every user does. > Similar as with assembly, you might have not taken a look at it, but there are people that do and could argue the same thing as you. ... No. The assembler is deterministic. Barring bugs, you can basically trust that it does exactly what it was told to. You absolutely cannot say the same of our beloved robot overlords.
joenot443 1y ago
"SwiftUI red circle with a white outline 100px" ``` Circle() .fill(Color.red) .overlay( Circle().stroke(Color.white, lineWidth: 4) ).frame(width: 100, height: 100) ``` Is the mapping 1:1 and completely lossless? Of course not, I'd say the former is most definitely a sort of abstraction of the latter, and one would be being disingenuous to pretend it's not.
KoolKat23 1y ago
It's still early stages, that is why. It is not yet good enough or there is not yet sufficient trust. Also there are still resources allocated to checking the code. I saw a post yesterday showing Brave browser's new tab using 70mb of RAM in the background. I'm very sure there's code there that can be optimized, but who gives a shit. It's splitting hairs and our computers are powerful enough now that it doesn't matter. Immateriality has abstracted that particular few line codes away.
- rafterydj 1y ago
  I feel like I'm taking crazy pills or misunderstanding you. Shouldn't it matter that they are using 70mb of RAM more or less totally wastefully? Maybe not a deal breaker for Brave, sure, but waste is waste. I understand the world is about compromises, but all the gains of essentially every computer program ever could be summed up by accumulation of small optimizations. Likewise, the accumulation of small wastes kills legacy projects more than anything else.
  - KoolKat23 1y ago
    Yes it can be construed as wasteful. But it's exactly that, a compromise. Could the programmer spend their time better elsewhere generating better value, not doing so is also wasteful. Supply and demand will decide what compromise is acceptable and what that compromise looks like.
  - Mtinie 1y ago
    It could matter but what isn't clear to me is if 70MB is wasteful in this specific context. Maybe? Maybe not? Flagging something as potentially problematic is useful but without additional information related to the tradeoffs being made this may be an optimized way to do whatever Brave is doing which requires the 70MB of RAM. Perhaps the non-optimal way it was previously doing it required 250MB of RAM and this is a significant improvement.
    - deleted 1y ago
      [deleted]
CuriouslyC 1y ago
I like to think of it as a duck-typed abstraction. I have formal specs which guarantee certain things about my software, and the agent has to satisfy those (including performance, etc). Given that, the code the agent generates is essentially fungible in the manner of machine code.
- sarchertech 1y ago
  Only if you can write specs precisely enough. For those of us who remember the way software used to be built, we learned that this is basically impossible and that English is a terrible language to even attempt it in. If you do make your specs precise enough, such that 2 different dev shops will produce functionally equivalent software, your specs are equivalent to code.
  - CuriouslyC 1y ago
    This is doable, I have a multi-stage process that makes it pretty reliable. Stage 1 is ideation, this can be with a LLM or humans, w/e, you just need a log. Stage 2 is conversion of that ideation log to a simple spec format that LLMs can write easily called SRF, which is fenced inside a nice markdown document humans can read and understand. You can edit that SRF if desired, have a conversation with the agent about it to get them to massage it, or just have your agent feed it into a tool I wrote which takes a SRF and converts it to a CUE with full formal validation and lots of other nice features. The value of this is that FOR FREE you can get comprehensive test defintions (unit+e2e), kube/terraform infra setup, documentation stubs, openai specs, etc. It's seriously magical.
    - sarchertech 1y ago
      Imagine that when you deploy you have an LLM that regenerates code based on your specs, since code is fungible as long as it fits the spec. Keeping in mind that I have seen hundreds to thousands of production errors in applications with very high coverage test suites? How many production errors would you expect to see over 5 years of LLM deployments.
theptip 1y ago
> and it never will be The only thing I’m certain of is that you’re highly overconfident. I’m sure plenty of assembly gurus said the same of the first compilers. > because it's not an abstraction This just seems like a category error. A human is not an abstraction, yet they write code and produce value. An IDE is a tool not an abstraction, yet they make humans more productive. When I talk about moving up the levels of abstraction I mean: taking on more abstract/less-concrete tasks. Instead of “please wire up login for our new prototype” it might be “please make the prototype fully production-ready, figure out what is needed” or even “please ship a new product to meet customer X’s need”.
- sarchertech 1y ago
  >“please ship a new product to meet customer X’s need” The customer would just ask the AI directly to meet their needs. They wouldn’t purchase the product from you.

elAhmo 1y ago

It is an abstraction. Just because you end up looking at what the prompt looks like “under the hood” in whichever language it produced the output, doesn’t mean every user does. Similar as with assembly, you might have not taken a look at it, but there are people that do and could argue the same thing as you. The lines will be very blurry in the near future.

card_zero 1y ago
I can fart in the general direction of the code and that's a kind of abstraction too. It distills my intent down to a simple raspberry noise which could then be interpreted by sufficiently intelligent software.
- joenot443 1y ago
  Sort of like when people argue about "what is art?", some folks find it clever to come up with the most bizarre of examples to point to, as if the entire concept rests on the existence of its members at the fringe. Personally, I think if your farts are an abstraction that you can derive useful meaning from the mapping, who are we to tell you no?
  - card_zero 1y ago
    So, can derive useful meaning from is the point of contention. (Also: bizarre examples = informative edge cases. Sometimes.)
- chaps 1y ago
  Could you please expand on this thought? I'm curious where this abstraction's inflection points are.
- Phemist 1y ago
  The final result of course depending on how many r's there are in the raspberry.
rsynnott 1y ago
> Just because you end up looking at what the prompt looks like “under the hood” in whichever language it produced the output, doesn’t mean every user does. > Similar as with assembly, you might have not taken a look at it, but there are people that do and could argue the same thing as you. ... No. The assembler is deterministic. Barring bugs, you can basically trust that it does exactly what it was told to. You absolutely cannot say the same of our beloved robot overlords.

joenot443 1y ago

"SwiftUI red circle with a white outline 100px" ``` Circle() .fill(Color.red) .overlay( Circle().stroke(Color.white, lineWidth: 4) ).frame(width: 100, height: 100) ``` Is the mapping 1:1 and completely lossless? Of course not, I'd say the former is most definitely a sort of abstraction of the latter, and one would be being disingenuous to pretend it's not.

KoolKat23 1y ago

It's still early stages, that is why. It is not yet good enough or there is not yet sufficient trust. Also there are still resources allocated to checking the code. I saw a post yesterday showing Brave browser's new tab using 70mb of RAM in the background. I'm very sure there's code there that can be optimized, but who gives a shit. It's splitting hairs and our computers are powerful enough now that it doesn't matter. Immateriality has abstracted that particular few line codes away.

rafterydj 1y ago
I feel like I'm taking crazy pills or misunderstanding you. Shouldn't it matter that they are using 70mb of RAM more or less totally wastefully? Maybe not a deal breaker for Brave, sure, but waste is waste. I understand the world is about compromises, but all the gains of essentially every computer program ever could be summed up by accumulation of small optimizations. Likewise, the accumulation of small wastes kills legacy projects more than anything else.
- KoolKat23 1y ago
  Yes it can be construed as wasteful. But it's exactly that, a compromise. Could the programmer spend their time better elsewhere generating better value, not doing so is also wasteful. Supply and demand will decide what compromise is acceptable and what that compromise looks like.
- Mtinie 1y ago
  It could matter but what isn't clear to me is if 70MB is wasteful in this specific context. Maybe? Maybe not? Flagging something as potentially problematic is useful but without additional information related to the tradeoffs being made this may be an optimized way to do whatever Brave is doing which requires the 70MB of RAM. Perhaps the non-optimal way it was previously doing it required 250MB of RAM and this is a significant improvement.
  - deleted 1y ago
    [deleted]

CuriouslyC 1y ago

I like to think of it as a duck-typed abstraction. I have formal specs which guarantee certain things about my software, and the agent has to satisfy those (including performance, etc). Given that, the code the agent generates is essentially fungible in the manner of machine code.

sarchertech 1y ago
Only if you can write specs precisely enough. For those of us who remember the way software used to be built, we learned that this is basically impossible and that English is a terrible language to even attempt it in. If you do make your specs precise enough, such that 2 different dev shops will produce functionally equivalent software, your specs are equivalent to code.
- CuriouslyC 1y ago
  This is doable, I have a multi-stage process that makes it pretty reliable. Stage 1 is ideation, this can be with a LLM or humans, w/e, you just need a log. Stage 2 is conversion of that ideation log to a simple spec format that LLMs can write easily called SRF, which is fenced inside a nice markdown document humans can read and understand. You can edit that SRF if desired, have a conversation with the agent about it to get them to massage it, or just have your agent feed it into a tool I wrote which takes a SRF and converts it to a CUE with full formal validation and lots of other nice features. The value of this is that FOR FREE you can get comprehensive test defintions (unit+e2e), kube/terraform infra setup, documentation stubs, openai specs, etc. It's seriously magical.
  - sarchertech 1y ago
    Imagine that when you deploy you have an LLM that regenerates code based on your specs, since code is fungible as long as it fits the spec. Keeping in mind that I have seen hundreds to thousands of production errors in applications with very high coverage test suites? How many production errors would you expect to see over 5 years of LLM deployments.

theptip 1y ago

> and it never will be The only thing I’m certain of is that you’re highly overconfident. I’m sure plenty of assembly gurus said the same of the first compilers. > because it's not an abstraction This just seems like a category error. A human is not an abstraction, yet they write code and produce value. An IDE is a tool not an abstraction, yet they make humans more productive. When I talk about moving up the levels of abstraction I mean: taking on more abstract/less-concrete tasks. Instead of “please wire up login for our new prototype” it might be “please make the prototype fully production-ready, figure out what is needed” or even “please ship a new product to meet customer X’s need”.

sarchertech 1y ago
>“please ship a new product to meet customer X’s need” The customer would just ask the AI directly to meet their needs. They wouldn’t purchase the product from you.