6 ms·

I'm glad that we're making progress towards a deeper understanding of what LLMs are inherently good at and what they're inherently bad at (not to say incapable

I'm glad that we're making progress towards a deeper understanding of what LLMs are inherently good at and what they're inherently bad at (not to say incapable of doing, but stuff that is less likely to work due to fundamental limitations). There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Or asking an LLM to write you the SQL query for your data analysis, rather than asking it to do your data analysis for you. What I'd really like to see is a more well defined taxonomy of work and studies on which bits work well with LLMs and which don't. I understand some of this intuitively, but am still building my intuition, and I see people tripping up on this all the time.

locknitpicker 6mo ago
> There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Not so long ago, this was how early adopters of LLM coding assistants claimed was the right way to use them in coding tasks: prompt to draft the outline, and then prompt to implement each function. There were even a few posts in HN on blogposts showing off this approach with terms inspired in animation work.
- danpalmer 6mo ago
  I'm not necessarily suggesting always getting down to literally the function level, although I think that gives you excellent quality control, but having a code-level understanding is clearly an important factor.
- nullsanity 6mo ago
  [dead]
- Sammi 6mo ago
  In short, LLMs are pretty great at working at a single level of abstraction at a time. You can go from the highest level and all the way down to the lowest level with LLMs, you just have to work at it iteratively one level at a time.
p-e-w 6mo ago
> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.
- dijit 6mo ago
  Character counting remains a huge issue without tools. Are you using only frontier models that are gated behind openai/anthropic/google APIs? Those use tools to help them out behind the scenes. It remains no less impressive, but I think we should be clear.
- rimliu 6mo ago
  of course, if you choose to ignore all the limitations they indeed have no limitations.
  - mkbosmans 6mo ago
    Nobody says they have no limitations. The question is are those limitation fundamental, i.e. can we expect improvement, say within a year.
    - danpalmer 6mo ago
      When I talk about fundamental limitations, I mean limitations that can't be solved, even if they could be improved. We have improved hallucinations significantly, and yet it seems clear that they are inherent to the technology and so will always exist to some extent.
      - p-e-w 6mo ago
        “Seems clear” based on what?
        6mo ago

locknitpicker 6mo ago

> There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Not so long ago, this was how early adopters of LLM coding assistants claimed was the right way to use them in coding tasks: prompt to draft the outline, and then prompt to implement each function. There were even a few posts in HN on blogposts showing off this approach with terms inspired in animation work.

danpalmer 6mo ago
I'm not necessarily suggesting always getting down to literally the function level, although I think that gives you excellent quality control, but having a code-level understanding is clearly an important factor.
nullsanity 6mo ago
[dead]
Sammi 6mo ago
In short, LLMs are pretty great at working at a single level of abstraction at a time. You can go from the highest level and all the way down to the lowest level with LLMs, you just have to work at it iteratively one level at a time.

p-e-w 6mo ago

> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.

dijit 6mo ago
Character counting remains a huge issue without tools. Are you using only frontier models that are gated behind openai/anthropic/google APIs? Those use tools to help them out behind the scenes. It remains no less impressive, but I think we should be clear.
rimliu 6mo ago
of course, if you choose to ignore all the limitations they indeed have no limitations.
- mkbosmans 6mo ago
  Nobody says they have no limitations. The question is are those limitation fundamental, i.e. can we expect improvement, say within a year.
  - danpalmer 6mo ago
    When I talk about fundamental limitations, I mean limitations that can't be solved, even if they could be improved. We have improved hallucinations significantly, and yet it seems clear that they are inherent to the technology and so will always exist to some extent.
    - p-e-w 6mo ago
      “Seems clear” based on what?
      - 6mo ago