3 ms·

Sorry for the tangent, but as a chart nerd it is so interesting to me the way that LLMs produce charts for articles like this - it's like they have no concept o

by ainch 19d ago

Sorry for the tangent, but as a chart nerd it is so interesting to me the way that LLMs produce charts for articles like this - it's like they have no concept of a 3rd party (the reader). So the writing and the vis gets overloaded with the particular context of the conversation, even if it would be irrelevant to a reader. Like the x-axis label of the first plot mentions that it shows gridlines every 8 ticks - I don't think that's a choice I've ever seen a person make. The next plot does the same thing, but in the title: "(dotted lines: evenly spaced reference)". Again, would anyone making a chart ever include such a specific detail in the title? Probably not beyond undergrad level. I encounter a similar thing when trying to work with LLMs to write articles - they cannot help including information or detail from the conversation, rather than empathising with the reader and filtering out what they might already know or not care about. They may be capable of solving decades-old maths problems, but on this particular axis they seem to be floundering at the level of a fresh graduate that's desperate to talk about what they've done rather than what their audience needs to know.

IanCal 19d ago
This is so tiring. > Like the x-axis label of the first plot mentions that it shows gridlines every 8 ticks - I don't think that's a choice I've ever seen a person make The spacing being 8 is important and seems not so bad to call this out in an extra place. Imagine LLMs didn’t exist for a moment. Is this the part of the article you’d find most intriguing? That the grid lines being an unusual distance apart is written both on the axis and in the description? Would it have seemed inhuman on a blog post about debugging a weird display issue on an e-reader? > They may be capable of solving decades-old maths problems, but on this particular axis they seem to be floundering at the level of a fresh graduate that's desperate to talk about what they've done rather than what their audience needs to know Have you ever worked with people beyond graduate level? Ever seen scientific posters at a conference?
- gong_hits 19d ago
  [dead]
- asdff 19d ago
  If I saw the charts in the pre AI era I would also be confused. I mean it is just awkward and not done like this. OP is right, no one on earth says they put gridlines every so and so ticks anywhere, much less the figure title. Gridlines should be obvious what they are trying to show on a chart like this. They should actually be aligned with the tick marks on the x axis. In fact, why are those on a nice even 10s scale while gridlines are on 8 ticks? If this was undergraduate work I saw at a conference, I'd guess they were hungover when they made it and didn't realize they made the tick marks and axis labels incongruent with eachother, and it should have been 10 marks. I'd still have no clue why they wrote that in the title. I'd guess that they'd never presented a figure like this and guessed what might be important to include. The fact that we are now in the ai era where figures like this come so easy with no thought is actually even more damning to the article author. How can I trust anything in this article is any good given that they thought these figures were fit to publish? Either they didn't proof the work or they didn't care, and both are pretty bad possibilities.
  - IanCal 19d ago
    > and it should have been 10 marks It absolutely should be 8 here. The fact that it is 8 is extremely relevant because the figure is showing a pattern that has that frequency.
- ainch 19d ago
  > Imagine LLMs didn’t exist for a moment. Is this the part of the article you’d find most intriguing? I would think it's odd! That's why I mentioned it. I like to pay attention to vis, I feel like I've seen many charts from Tufte-y perfection, to slapdash undergrad presentations, and to experienced consultants banging things together in Excel at 1am. This tic feels very LLM-y to me, which imo makes it interesting to think about - how does it arise? I also think this failure mode has been resistant to model upgrades in a way that mathematical reasoning has not, which I also find interesting as someone in ML research. It feels like this kind of mentalisation or theory of mind is a capability which isn't obviously elicited from the frontier labs' crop of RLVR tasks. > Have you ever worked with people beyond graduate level? Ever seen scientific posters at a conference? I've worked with a fair number of people beyond graduate level. In previous jobs I've run teams and hired people - hence my comment about graduates. We used to have an interview stage where candidates presented some simple data analysis, and it often centred on what they had done rather than what was most worth knowing. I have also attended scientific conferences, in fact I was at one a week ago! Is there a particular inference you would like me to make from those experiences?
  - IanCal 19d ago
    I want to really understand this, your issue is that the non-standard grid spacing is called out on the axis? The grid spacing being 8 is unusual but clearly sensible here - your issue is just that this is written also on the axis label? > Is there a particular inference you would like me to make from those experiences? The incredible amount of unnecessary detail people put into their posters specifically (hence the better poster movement), desperate to show everything.
    - ainch 18d ago
      > I want to really understand this, your issue is that the non-standard grid spacing is called out on the axis? The grid spacing being 8 is unusual but clearly sensible here - your issue is just that this is written also on the axis label? Pretty much yep - it struck me as odd and reminded me of an LLM tic I've been thinking about recently, so I thought I'd write a short comment. > The incredible amount of unnecessary detail people put into their posters specifically (hence the better poster movement), desperate to show everything. I hadn't heard of the better poster idea before, thanks for clueing me in. I largely agree that posters have a lot of unnecessary detail, but I think it's a different flavour to the LLM stuff - more like they're so excited to tell you everything in their paper. Whereas LLMs focus on odd details, or fail to explain things. Can I ask, if you use LLMs often in your work, have you never run into this experience?
stratts 19d ago
This happens with everything LLMs write - code comments, readmes, user facing UI text, etc. It just can't help but include every irrelevant detail from the context. It's definitely gotten worse over time as well. Looking at older README files I generated using Opus/Sonnet 4.5, its actually quite shocking how clear and concise they are. Whereas eg, Opus 5 will write paragraphs and paragraphs about random things I prompted, or a partial solution that was then scrapped for something else. My theory is that the increasing strength in performing long-context tasks has led to this particular quirk.
- viccis 19d ago
  >It just can't help but include every irrelevant detail from the context. The worst is when it walked through a dozen or so questions, and it decides to litter comments with "Per answer 7, ..." with no further context.
  - black_knight 18d ago
    I have in my Claude.md that comments must be diegetic, if needed at all. This seems to have helped. And it is then something which gets checked by agents in code review.
    - killix 17d ago
      [flagged]
alansaber 18d ago
Perfectly stated and also explains why it slips past the author (they just nod and let it slip pass).

by ainch 19d ago

IanCal 19d ago

This is so tiring. > Like the x-axis label of the first plot mentions that it shows gridlines every 8 ticks - I don't think that's a choice I've ever seen a person make The spacing being 8 is important and seems not so bad to call this out in an extra place. Imagine LLMs didn’t exist for a moment. Is this the part of the article you’d find most intriguing? That the grid lines being an unusual distance apart is written both on the axis and in the description? Would it have seemed inhuman on a blog post about debugging a weird display issue on an e-reader? > They may be capable of solving decades-old maths problems, but on this particular axis they seem to be floundering at the level of a fresh graduate that's desperate to talk about what they've done rather than what their audience needs to know Have you ever worked with people beyond graduate level? Ever seen scientific posters at a conference?

gong_hits 19d ago
[dead]
asdff 19d ago
If I saw the charts in the pre AI era I would also be confused. I mean it is just awkward and not done like this. OP is right, no one on earth says they put gridlines every so and so ticks anywhere, much less the figure title. Gridlines should be obvious what they are trying to show on a chart like this. They should actually be aligned with the tick marks on the x axis. In fact, why are those on a nice even 10s scale while gridlines are on 8 ticks? If this was undergraduate work I saw at a conference, I'd guess they were hungover when they made it and didn't realize they made the tick marks and axis labels incongruent with eachother, and it should have been 10 marks. I'd still have no clue why they wrote that in the title. I'd guess that they'd never presented a figure like this and guessed what might be important to include. The fact that we are now in the ai era where figures like this come so easy with no thought is actually even more damning to the article author. How can I trust anything in this article is any good given that they thought these figures were fit to publish? Either they didn't proof the work or they didn't care, and both are pretty bad possibilities.
- IanCal 19d ago
  > and it should have been 10 marks It absolutely should be 8 here. The fact that it is 8 is extremely relevant because the figure is showing a pattern that has that frequency.
ainch 19d ago
> Imagine LLMs didn’t exist for a moment. Is this the part of the article you’d find most intriguing? I would think it's odd! That's why I mentioned it. I like to pay attention to vis, I feel like I've seen many charts from Tufte-y perfection, to slapdash undergrad presentations, and to experienced consultants banging things together in Excel at 1am. This tic feels very LLM-y to me, which imo makes it interesting to think about - how does it arise? I also think this failure mode has been resistant to model upgrades in a way that mathematical reasoning has not, which I also find interesting as someone in ML research. It feels like this kind of mentalisation or theory of mind is a capability which isn't obviously elicited from the frontier labs' crop of RLVR tasks. > Have you ever worked with people beyond graduate level? Ever seen scientific posters at a conference? I've worked with a fair number of people beyond graduate level. In previous jobs I've run teams and hired people - hence my comment about graduates. We used to have an interview stage where candidates presented some simple data analysis, and it often centred on what they had done rather than what was most worth knowing. I have also attended scientific conferences, in fact I was at one a week ago! Is there a particular inference you would like me to make from those experiences?
- IanCal 19d ago
  I want to really understand this, your issue is that the non-standard grid spacing is called out on the axis? The grid spacing being 8 is unusual but clearly sensible here - your issue is just that this is written also on the axis label? > Is there a particular inference you would like me to make from those experiences? The incredible amount of unnecessary detail people put into their posters specifically (hence the better poster movement), desperate to show everything.
  - ainch 18d ago
    > I want to really understand this, your issue is that the non-standard grid spacing is called out on the axis? The grid spacing being 8 is unusual but clearly sensible here - your issue is just that this is written also on the axis label? Pretty much yep - it struck me as odd and reminded me of an LLM tic I've been thinking about recently, so I thought I'd write a short comment. > The incredible amount of unnecessary detail people put into their posters specifically (hence the better poster movement), desperate to show everything. I hadn't heard of the better poster idea before, thanks for clueing me in. I largely agree that posters have a lot of unnecessary detail, but I think it's a different flavour to the LLM stuff - more like they're so excited to tell you everything in their paper. Whereas LLMs focus on odd details, or fail to explain things. Can I ask, if you use LLMs often in your work, have you never run into this experience?

stratts 19d ago

This happens with everything LLMs write - code comments, readmes, user facing UI text, etc. It just can't help but include every irrelevant detail from the context. It's definitely gotten worse over time as well. Looking at older README files I generated using Opus/Sonnet 4.5, its actually quite shocking how clear and concise they are. Whereas eg, Opus 5 will write paragraphs and paragraphs about random things I prompted, or a partial solution that was then scrapped for something else. My theory is that the increasing strength in performing long-context tasks has led to this particular quirk.

viccis 19d ago
>It just can't help but include every irrelevant detail from the context. The worst is when it walked through a dozen or so questions, and it decides to litter comments with "Per answer 7, ..." with no further context.
- black_knight 18d ago
  I have in my Claude.md that comments must be diegetic, if needed at all. This seems to have helped. And it is then something which gets checked by agents in code review.
  - killix 17d ago
    [flagged]

alansaber 18d ago

Perfectly stated and also explains why it slips past the author (they just nod and let it slip pass).