6 ms·

There's the old quote from Babbage: > On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers

There's the old quote from Babbage: > On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time. It is kind of impressive how I'll ask for some code in the dumbest, vaguest, sometimes even wrong way, but so long as I have the proper context built up, I can get something pretty close to what I actually wanted. Though I still have problems where I can ask as precisely as possible and get things not even close to what I'm looking for.

CobrastanJorji 1y ago
We wanted to check the clock at the wrong time but read the correct time. Since a broken clock is right twice a day, we broke the clock, which solves our problem some of the time!
- pca006132 1y ago
  The nice thing is that a fully broken clock is accurate more often than a slightly deviated clock.
  - teddyh 1y ago
    Only if the deviated clock is fast. If a clock is, instead, slow, it is correct more often than a stopped clock.
  - antifa 1y ago
    A clock that's 5 seconds, 5 minutes, or 5 hours ahead, or counts an hour as 61 minutes, is still more useful than a clock that does not move it's hands at all.
meowface 1y ago
It is fun to watch. I've sometimes indeed seen the LLM say something like "I'm assuming you meant [X]".
godelski 1y ago
How do you know the code is right?
- ic_fly2 1y ago
  The LLM generated unit tests pass. Obviously!
  - lazide 1y ago
    Just don’t look at the generated unit tests, and we’re fine.
  - godelski 1y ago
    It seems most people are making this answer but without the sarcasm...
- dkdbejwi383 1y ago
  If customers don’t complain it must be working
  - godelski 1y ago
    You don't hear the complaints. That's different than no complaints. Trust me, they got them. I got plenty of complaints for Apple, Google, Netflix, and everyone else. Shit that can be fixed with just a fucking regex. Here's an example: my gf is duplicated in my Apple contacts. It can't find the duplicate, despite same name, nickname, phone number, email, and birthday. Which there's three entries on my calendar for her birthday. Guess what happened when I manually merged? She now has 4(!!!!!) entries!! How the fuck does that increase! Trust me, they complain, you just don't listen
- fsloth 1y ago
  The program behaves as you want. No, really - there is tons of potentially value-adding code that can be of throwaway quality just as long as it’s zero effort to write it. Design explorations, refactorings, erc etc.
nitwit005 1y ago
It's very impressive that I can type misheard song lyrics into Google, and yet still have the right song pop up. But, having taken a chance to look at the raw queries people type into apps, I'm afraid neither machine nor human is going to make sense of a lot of it.
ivape 1y ago
We're talking about God function. function God (any param you can think of) { }
CrimsonRain 1y ago
theseday,s i ofen donot correct my typos even wheni notice them while cahtting with LLMS. So far 0 issues.
kibwen 1y ago
> This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time. This is not the point of that Babbage quote, and no, LLMs have not solved it, because it cannot be solved, because "garbage in, garbage out" is a fundamental observation of the limits of logic itself, having more to with the laws of thermodynamics than it does with programming. The output of a logical process cannot be more accurate than the inputs to that process; you cannot conjure information out of the ether. The LLM isn't the logical process in this analogy, it's one of the inputs.
- rcxdude 1y ago
  At a fundamental level, yes, and even in human-to-human interaction this kind of thing happens all the time. The difference is that humans are generally quite good at resolving most ambiguities and contradictions in a request correctly and implicitly (sometimes surprisingly bad at doing so explicitly!). Which is why human language tends to be more flexible and expressive than programming languages (but bad at precision). LLMs basically can do some of the same thing, so you don't need to specify all the 'obvious' implicit details.
  - Kye 1y ago
    Handing an LLM a file and asking it to extract data out of it with no further context or explanation of what I'm looking for with good results does feel a bit like the future. I still do add context just to get more consistent results, but it's neat that LLMs handle fuzzy queries as well as they do.
  - kibwen 1y ago
    The Babbage anecdote isn't about ambiguous inputs, it's about wrong inputs. Imagine wanting to know the answer to 2+2, so you go up to the machine and ask "What is 3+3?", expecting that it will tell you what 2+2 is. Adding an LLM as input to this process (along with an implicit acknowledgement that you're uncertain about your inputs) might produce a response "Are you sure you didn't mean to ask what 2+2 is?", but that's because the LLM is a big ball of likelihoods and it's more common to ask for 2+2 than for 3+3. But it's not magic; the LLM cannot operate on information that it was not given, rather it's that a lot of the information that it has was given to it during training. It's no more a breakthrough of fundamental logic than Google showing you results for "air fryer" when you type in "air frier".
jajko 1y ago
Well, you can enter 4-5 relatively vague keywords into google and first/second stackoverflow link will probably provide plenty of relevant code. Given that, its much less impressive since >95% of the problems and queries just keep repeating.

CobrastanJorji 1y ago

We wanted to check the clock at the wrong time but read the correct time. Since a broken clock is right twice a day, we broke the clock, which solves our problem some of the time!

pca006132 1y ago
The nice thing is that a fully broken clock is accurate more often than a slightly deviated clock.
- teddyh 1y ago
  Only if the deviated clock is fast. If a clock is, instead, slow, it is correct more often than a stopped clock.
- antifa 1y ago
  A clock that's 5 seconds, 5 minutes, or 5 hours ahead, or counts an hour as 61 minutes, is still more useful than a clock that does not move it's hands at all.

meowface 1y ago

It is fun to watch. I've sometimes indeed seen the LLM say something like "I'm assuming you meant [X]".

godelski 1y ago

How do you know the code is right?

ic_fly2 1y ago
The LLM generated unit tests pass. Obviously!
- lazide 1y ago
  Just don’t look at the generated unit tests, and we’re fine.
- godelski 1y ago
  It seems most people are making this answer but without the sarcasm...
dkdbejwi383 1y ago
If customers don’t complain it must be working
- godelski 1y ago
  You don't hear the complaints. That's different than no complaints. Trust me, they got them. I got plenty of complaints for Apple, Google, Netflix, and everyone else. Shit that can be fixed with just a fucking regex. Here's an example: my gf is duplicated in my Apple contacts. It can't find the duplicate, despite same name, nickname, phone number, email, and birthday. Which there's three entries on my calendar for her birthday. Guess what happened when I manually merged? She now has 4(!!!!!) entries!! How the fuck does that increase! Trust me, they complain, you just don't listen
fsloth 1y ago
The program behaves as you want. No, really - there is tons of potentially value-adding code that can be of throwaway quality just as long as it’s zero effort to write it. Design explorations, refactorings, erc etc.

nitwit005 1y ago

It's very impressive that I can type misheard song lyrics into Google, and yet still have the right song pop up. But, having taken a chance to look at the raw queries people type into apps, I'm afraid neither machine nor human is going to make sense of a lot of it.

ivape 1y ago

We're talking about God function. function God (any param you can think of) { }

CrimsonRain 1y ago

theseday,s i ofen donot correct my typos even wheni notice them while cahtting with LLMS. So far 0 issues.

kibwen 1y ago

> This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time. This is not the point of that Babbage quote, and no, LLMs have not solved it, because it cannot be solved, because "garbage in, garbage out" is a fundamental observation of the limits of logic itself, having more to with the laws of thermodynamics than it does with programming. The output of a logical process cannot be more accurate than the inputs to that process; you cannot conjure information out of the ether. The LLM isn't the logical process in this analogy, it's one of the inputs.

rcxdude 1y ago
At a fundamental level, yes, and even in human-to-human interaction this kind of thing happens all the time. The difference is that humans are generally quite good at resolving most ambiguities and contradictions in a request correctly and implicitly (sometimes surprisingly bad at doing so explicitly!). Which is why human language tends to be more flexible and expressive than programming languages (but bad at precision). LLMs basically can do some of the same thing, so you don't need to specify all the 'obvious' implicit details.
- Kye 1y ago
  Handing an LLM a file and asking it to extract data out of it with no further context or explanation of what I'm looking for with good results does feel a bit like the future. I still do add context just to get more consistent results, but it's neat that LLMs handle fuzzy queries as well as they do.
- kibwen 1y ago
  The Babbage anecdote isn't about ambiguous inputs, it's about wrong inputs. Imagine wanting to know the answer to 2+2, so you go up to the machine and ask "What is 3+3?", expecting that it will tell you what 2+2 is. Adding an LLM as input to this process (along with an implicit acknowledgement that you're uncertain about your inputs) might produce a response "Are you sure you didn't mean to ask what 2+2 is?", but that's because the LLM is a big ball of likelihoods and it's more common to ask for 2+2 than for 3+3. But it's not magic; the LLM cannot operate on information that it was not given, rather it's that a lot of the information that it has was given to it during training. It's no more a breakthrough of fundamental logic than Google showing you results for "air fryer" when you type in "air frier".

jajko 1y ago

Well, you can enter 4-5 relatively vague keywords into google and first/second stackoverflow link will probably provide plenty of relevant code. Given that, its much less impressive since >95% of the problems and queries just keep repeating.