14 ms·

The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. T

The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges. LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.

programd 1y ago
> [LLMs] spit out the most likely text to follow some other text based on probability. Modern coding AI models are not just probability crunching transformers. They haven't been just that for some time. In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. The point is that the entire expert system is getting better at a rapid pace and the probability bit is just one part of it. The complexity frontier for code generation keeps moving and there's still a lot of low hanging fruit to be had in pushing it forward. > They're great for writing boilerplate that has been written a million times with different variations That's >90% of all code in the wild. Probably more. We have three quarters of a century of code in our history so there is very little that's original anymore. Maybe original to the human coder fresh out of school, but the models have all this history to draw upon. So if the models produce the boilerplate reliably then human toil in writing if/then statements is at an end. Kind of like - barring the occasional mad genious [0] - the vast majority of coders don't write assembly to create a website anymore. [0] https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index.txt https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index...
- Night_Thastus 1y ago
  >In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. This is lipstick on a pig. All those methods are impressive, but ultimately workarounds for an idea that is fundamentally unsuitable for programming. >That's >90% of all code in the wild. Probably more. Maybe, but not 90% of time spent on programming. Boilerplate is easy. It's the 20%/80% rule in action. I don't deny these tools can be useful and save time - but they can't be left to their own devices. They need to be tightly controlled and given narrow scopes, with heavy oversight by an SME who knows what the code is supposed to be doing. "Design W module with X interface designed to do Y in Z way", keeping it as small as possible and reviewing it to hell and back. And keeping it accountable by making tests yourself. Never let it test itself, it simply cannot be trusted to do so. LLMs are incredibly good at writing something that looks reasonable, but is complete nonsense. That's horrible from a code maintenance perspective.
- mgaunard 1y ago
  Except we should aim to reduce the boilerplate through good design, instead of creating more of it on an industrial scale.
  - exe34 1y ago
    what we should and what we are forced to do are very different things. if I can get a machine to do the stuff I hate dealing with, I'll take it every time.
    - mgaunard 1y ago
      who's going to be held accountable when the boilerplate fails? the AI?
      - exe34 1y ago
exe34 1y ago
I take it you haven't tried an LLM in a few years?
- Night_Thastus 1y ago
  Just a couple of weeks ago on mid-range models. The problem is not implementation or refinement - the core idea is fundamentally flawed.
  - nativeit 1y ago
    The problem is we’re now arguing with religious zealots. I am not being sarcastic.
    - gibbitz 1y ago
      I feel this way about Typescript too. There are a lot of people in engineering these days who don't think critically or exercise full observation when using popular technologies. I don't feel like it was like this 15 years ago, but it probably was...
    - exe34 1y ago
      that's correct. those who believe only carbon can achieve intelligence.
      - melagonster 1y ago
        Yes, Carbon do not give them human rights.
      - shkkmo 1y ago
        If you stereotype the people who disagree with you, you'll have a very hard time understanding their actual arguments.
        exe34 1y ago
        I stopped finding those arguments entertaining after a while. It always ends up "there's something that will always be missing, I just know it, but I won't tell you what. I'm just willing to go round and round in circles."
bitwize 1y ago
> LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. Said like a true software person. I'm to understand that computer people are looking at LLMs from the wrong end of the telescope; and that from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs. The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on.
- Night_Thastus 1y ago
  >from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs Hahahahahaha. Oh god, you're serious. Sure, let's just completely ignore all the other types of processing that the brain does. Sensory input processing, emotional regulation, social behavior, spatial reasoning, long and short term planning, the complex communication and feedback between every part of the body - even down to the gut microbiome. The brain (human or otherwise) is incredibly complex and we've barely scraped the surface of how it works. It's not just nuerons (which are themselves complex), it's interactions between thousands of types of cells performing multiple functions each. It will likely be hundreds of years before we get a full grasp on how it truly works - if we ever do at all.
  - N_Lens 1y ago
    Reminds me of that old chestnut - “if the human brain were simpler to understand, we wouldn’t be smart enough to do so”
- fzeroracer 1y ago
  > The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on. This is trivially proven false, because LLMs have far larger memory than your average human brain and are trained on far more data. Yet they do not come even close to approximating human cognition.
motorest 1y ago
> For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. Not to disagree, but "non-english" isn't exactly relevant. For unique problems, LLMs can still manage to output hallucinations that end up being right or useful. For example, LLMs can predict what an API looks like and how it works even if they do not have the API in context if the API was designed following standard design principles and best practices. LLMs can also build up context while you interact with them, which means that iteratively prompting them that X works while Y doesn't will help them build the necessary and sufficient context to output accurate responses.
- windward 1y ago
  >hallucinations This is the first word that came to mind when reading the comment above yours. Like: >They can't, despite marketing, really reason They aren't, despite marketing, really hallucinations. Now I understand why these companies don't want to market using terms like "extrapolated bullshit", but I don't understand how there is any technological solution to it without starting from a fresh base.
  - motorest 1y ago
    > They aren't, despite marketing, really hallucinations. They are hallucinations. You might not be aware of what that concept means in terms of LLMs but just because you are oblivious to the definition of a concept that does not mean it doesn't exist. You can learn about the concept by spending a couple of minutes reading this article on Wikipedia. https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence) https://en.wikipedia.org/wiki/Hallucination_(artificial_inte... > Now I understand why these companies don't want to market using terms like "extrapolated bullshit", (...) That's literally in the definition. Please do yourself a favour and get acquainted with the topic before posting comments.
    - windward 1y ago
      That page was made in December 2022, requires specifying '(artificial intelligence)' and says: >(also called bullshitting,[1][2] confabulation,[3] or delusion)[4] Here's the first linked source: https://www.psypost.org/scholars-ai-isnt-hallucinating-its-bullshitting/ https://www.psypost.org/scholars-ai-isnt-hallucinating-its-b...
mfbx9da4 1y ago
How can you tell a human actually understands? Prove to me that human thought is not predicting the most probable next token. If it quacks like duck. In psychology research the only way to research if a human is happy is to ask them.
- alpaca128 1y ago
  Does speaking in your native language, speaking in a second language, thinking about your life and doing maths feel exactly the same to you? > Prove to me that human thought is not predicting the most probable next token. Explain the concept of color to a completely blind person. If their brain does nothing but process tokens this should be easy. > How can you tell a human actually understands? What a strange question coming from a human. I would say if you are a human with a consciousness you are able to answer this for yourself, and if you aren't no answer will help.
  - randallsquared 1y ago
    > What a strange question coming from a human. Oh, I dunno. The whole "mappers vs packers" and "wordcels vs shape rotators" dichotomies point at an underlying truth, which is that humans don't always actually understand what they're talking about, even when they're saying all the "right" words. This is one reason why tech interviewing is so difficult: it's partly a task of figuring out if someone understands, or has just learned the right phrases and superficial exercises.
- 0xak 1y ago
  That is ill-posed. Take any algorithm at all, e.g. a TSP solver. Make a "most probable next token predictor" that takes the given traveling salesman problem, runs the solver, and emits the first token of the solution, then reruns the solver and emits the next token, and so on. By this thought experiment you can make any computational process into "predict the most probable next token" - at an extreme runtime cost. But if you do so, you arguably empty the concept "token predictor" of most of its meaning. So you would need to more accurately specify what you mean by a token predictor so that the answer isn't trivially true (for every kind of thought that's computation-like).

programd 1y ago

> [LLMs] spit out the most likely text to follow some other text based on probability. Modern coding AI models are not just probability crunching transformers. They haven't been just that for some time. In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. The point is that the entire expert system is getting better at a rapid pace and the probability bit is just one part of it. The complexity frontier for code generation keeps moving and there's still a lot of low hanging fruit to be had in pushing it forward. > They're great for writing boilerplate that has been written a million times with different variations That's >90% of all code in the wild. Probably more. We have three quarters of a century of code in our history so there is very little that's original anymore. Maybe original to the human coder fresh out of school, but the models have all this history to draw upon. So if the models produce the boilerplate reliably then human toil in writing if/then statements is at an end. Kind of like - barring the occasional mad genious [0] - the vast majority of coders don't write assembly to create a website anymore. [0] https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index.txt https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index...

Night_Thastus 1y ago
>In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago. This is lipstick on a pig. All those methods are impressive, but ultimately workarounds for an idea that is fundamentally unsuitable for programming. >That's >90% of all code in the wild. Probably more. Maybe, but not 90% of time spent on programming. Boilerplate is easy. It's the 20%/80% rule in action. I don't deny these tools can be useful and save time - but they can't be left to their own devices. They need to be tightly controlled and given narrow scopes, with heavy oversight by an SME who knows what the code is supposed to be doing. "Design W module with X interface designed to do Y in Z way", keeping it as small as possible and reviewing it to hell and back. And keeping it accountable by making tests yourself. Never let it test itself, it simply cannot be trusted to do so. LLMs are incredibly good at writing something that looks reasonable, but is complete nonsense. That's horrible from a code maintenance perspective.
mgaunard 1y ago
Except we should aim to reduce the boilerplate through good design, instead of creating more of it on an industrial scale.
- exe34 1y ago
  what we should and what we are forced to do are very different things. if I can get a machine to do the stuff I hate dealing with, I'll take it every time.
  - mgaunard 1y ago
    who's going to be held accountable when the boilerplate fails? the AI?
    - exe34 1y ago

exe34 1y ago

I take it you haven't tried an LLM in a few years?

Night_Thastus 1y ago
Just a couple of weeks ago on mid-range models. The problem is not implementation or refinement - the core idea is fundamentally flawed.
- nativeit 1y ago
  The problem is we’re now arguing with religious zealots. I am not being sarcastic.
  - gibbitz 1y ago
    I feel this way about Typescript too. There are a lot of people in engineering these days who don't think critically or exercise full observation when using popular technologies. I don't feel like it was like this 15 years ago, but it probably was...
  - exe34 1y ago
    that's correct. those who believe only carbon can achieve intelligence.
    - melagonster 1y ago
      Yes, Carbon do not give them human rights.
    - shkkmo 1y ago
      If you stereotype the people who disagree with you, you'll have a very hard time understanding their actual arguments.
      - exe34 1y ago
        I stopped finding those arguments entertaining after a while. It always ends up "there's something that will always be missing, I just know it, but I won't tell you what. I'm just willing to go round and round in circles."

bitwize 1y ago

> LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write. Said like a true software person. I'm to understand that computer people are looking at LLMs from the wrong end of the telescope; and that from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs. The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on.

Night_Thastus 1y ago
>from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs Hahahahahaha. Oh god, you're serious. Sure, let's just completely ignore all the other types of processing that the brain does. Sensory input processing, emotional regulation, social behavior, spatial reasoning, long and short term planning, the complex communication and feedback between every part of the body - even down to the gut microbiome. The brain (human or otherwise) is incredibly complex and we've barely scraped the surface of how it works. It's not just nuerons (which are themselves complex), it's interactions between thousands of types of cells performing multiple functions each. It will likely be hundreds of years before we get a full grasp on how it truly works - if we ever do at all.
- N_Lens 1y ago
  Reminds me of that old chestnut - “if the human brain were simpler to understand, we wouldn’t be smart enough to do so”
fzeroracer 1y ago
> The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on. This is trivially proven false, because LLMs have far larger memory than your average human brain and are trained on far more data. Yet they do not come even close to approximating human cognition.

motorest 1y ago

> For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model. Not to disagree, but "non-english" isn't exactly relevant. For unique problems, LLMs can still manage to output hallucinations that end up being right or useful. For example, LLMs can predict what an API looks like and how it works even if they do not have the API in context if the API was designed following standard design principles and best practices. LLMs can also build up context while you interact with them, which means that iteratively prompting them that X works while Y doesn't will help them build the necessary and sufficient context to output accurate responses.

windward 1y ago
>hallucinations This is the first word that came to mind when reading the comment above yours. Like: >They can't, despite marketing, really reason They aren't, despite marketing, really hallucinations. Now I understand why these companies don't want to market using terms like "extrapolated bullshit", but I don't understand how there is any technological solution to it without starting from a fresh base.
- motorest 1y ago
  > They aren't, despite marketing, really hallucinations. They are hallucinations. You might not be aware of what that concept means in terms of LLMs but just because you are oblivious to the definition of a concept that does not mean it doesn't exist. You can learn about the concept by spending a couple of minutes reading this article on Wikipedia. https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence) https://en.wikipedia.org/wiki/Hallucination_(artificial_inte... > Now I understand why these companies don't want to market using terms like "extrapolated bullshit", (...) That's literally in the definition. Please do yourself a favour and get acquainted with the topic before posting comments.
  - windward 1y ago
    That page was made in December 2022, requires specifying '(artificial intelligence)' and says: >(also called bullshitting,[1][2] confabulation,[3] or delusion)[4] Here's the first linked source: https://www.psypost.org/scholars-ai-isnt-hallucinating-its-bullshitting/ https://www.psypost.org/scholars-ai-isnt-hallucinating-its-b...

mfbx9da4 1y ago

How can you tell a human actually understands? Prove to me that human thought is not predicting the most probable next token. If it quacks like duck. In psychology research the only way to research if a human is happy is to ask them.

alpaca128 1y ago
Does speaking in your native language, speaking in a second language, thinking about your life and doing maths feel exactly the same to you? > Prove to me that human thought is not predicting the most probable next token. Explain the concept of color to a completely blind person. If their brain does nothing but process tokens this should be easy. > How can you tell a human actually understands? What a strange question coming from a human. I would say if you are a human with a consciousness you are able to answer this for yourself, and if you aren't no answer will help.
- randallsquared 1y ago
  > What a strange question coming from a human. Oh, I dunno. The whole "mappers vs packers" and "wordcels vs shape rotators" dichotomies point at an underlying truth, which is that humans don't always actually understand what they're talking about, even when they're saying all the "right" words. This is one reason why tech interviewing is so difficult: it's partly a task of figuring out if someone understands, or has just learned the right phrases and superficial exercises.
0xak 1y ago
That is ill-posed. Take any algorithm at all, e.g. a TSP solver. Make a "most probable next token predictor" that takes the given traveling salesman problem, runs the solver, and emits the first token of the solution, then reruns the solver and emits the next token, and so on. By this thought experiment you can make any computational process into "predict the most probable next token" - at an extreme runtime cost. But if you do so, you arguably empty the concept "token predictor" of most of its meaning. So you would need to more accurately specify what you mean by a token predictor so that the answer isn't trivially true (for every kind of thought that's computation-like).