5 ms·

This to me reads like a poignant commentary on the catastrophic loss of human agency, with the actual commit being highly revealing [0]. Author wants to hide a

This to me reads like a poignant commentary on the catastrophic loss of human agency, with the actual commit being highly revealing [0]. Author wants to hide a horizontal scrollbar. Any junior frontend dev worth their salt will be asking right away "where do I stick `overflow-x: hidden;`?" A complete solution will then require hitting "Inspect element" in the browser to find the CSS class and running (rip)grep to find where it is in code, to then add a single line to. An actual proactive programmer might start asking more pointed questions like what content does an empty textbox have that it overflows? And why do I need to insert this workaround that treats the symptom and not the root cause in two different places? Isn't it better to style `textarea` once? Etc, etc. [0] https://github.com/datasette/datasette-agent/commit/a75a8b727b42c30ced1fc41dc8add7eb9f04fefe https://github.com/datasette/datasette-agent/commit/a75a8b72...

gib444 4mo ago
The 'better' fixes are often for our (human) benefit. These messy fixes serve the AI companies' interests of creating messes that need even more tokens (money) later. Bad and self-serving developers also act the same, creating tech debt
piker 4mo ago
This is exactly right. By offloading this trivial task to the LLM, Simon has abandoned the opportunity to evaluate the abstraction with additional information and improve it. Instead, we let the agent spend $12 and make the fix while learning nothing.
- discordance 4mo ago
  I see it as a prioritization exercise. I know the above is a trivial example, but more generally, does the guy who wrote Datasette and Django want to wrangle front end and css, or do they want to work on something else?
  - smartbit 4mo ago
    See above https://news.ycombinator.com/item?id=48498573#48502311 https://news.ycombinator.com/item?id=48498573#48502311
- oulipo2 4mo ago
  [flagged]
  - beernet 4mo ago
    [flagged]
    - harperlee 4mo ago
      It was posted at 5am in New York... not sure that that was a US view, so the fact that the platform is US-owned doesn't seem so relevant, if there's a global audience. That being said, I do agree it is a legit thought (and moreso, completely on point in the subthread discussing downsides), and that it shouldn't be downvoted.
    - user43928 4mo ago
      While one can raise environmental concerns about the AI datacenter buildout, I don't think it is fair to say that it "ruins the planet". I don't think it is a good contribution to the discussion around Simon's LLM use to fix a CSS bug.
    - _heimdall 4mo ago
      That's an interesting choice as a source. It doesn't mention climate change or human impacts at all and describes El Niño as a naturally occurring event. > The El Nino is a phenomenon that occurs naturally
      - brickers 4mo ago
        El Niño is a naturally occurring event
      - dahart 4mo ago
        El Niño has been occurring naturally for more than 10,000 years. https://en.wikipedia.org/wiki/El_Ni%C3%B1o%E2%80%93Southern_Oscillation#History https://en.wikipedia.org/wiki/El_Ni%C3%B1o%E2%80%93Southern_...
- jmmcd 4mo ago
  People are missing that Willison is among the very best people we have in the role of (for lack of a good name): early access to frontier models, evaluate them in real scenarios, no wishful thinking, hype, or doom, communicate the possibilities. Yes he could have fixed this himself but then he would have learned nothing about the AI, and we wouldn't have read a fascinating and important article.
  - risyachka 4mo ago
    >> he would have learned nothing about the AI there is absolutely zero value in spending time to learn about new models as in few months new model will be out and whatever you learned about the current one will be useless. Also with models getting better and better you have to know less and less to achieve same results.
    - simonw 4mo ago
      My experience has been the exact opposite. As the models get better you need to know more about their capabilities, because otherwise you risk prompting Claude Fable 5 like it's GPT-4o and complaining loudly about how it's all hype and nothing about these models is improving at all (yes, I do see people say that.) Getting the best results out of these models requires skill, experience, intuition, and domain expertise. There's always room for improving every one of those.
      - ViscountPenguin 4mo ago
        Eh, I've have the exact opposite experience. Way back before instruct models it was pretty difficult, but for the last couple of years I haven't needed anything more complex than the type of text that I might send in a detailed email to a colleague.
      - philipwhiuk 4mo ago
        Isn't the whole point of a better model that it should be better at understanding you than the previous one? So the same prompt should return a better answer. Prompting differently to the new model seems entirely backwards when trying to determine if the model has improved.
- simonw 4mo ago
  Things I learned from this: - Fable will do a whole lot more than you might expect in order to verify a fix. I learned that it's "relentlessly proactive". That's a good title for a blog entry! - You can take screenshots of a window in macOS using the "screencapture" CLI command, but you'll need the integer window ID first. - That windowID is accessible via "Quartz.CGWindowListCopyWindowInfo(Quartz.kCGWindowListOptionOnScreenOnly, Quartz.kCGNullWindowID)" using the pyobjc-framework-Quartz library, which installs cleanly via "uv run". - A neat trick for simulating keyboard shortcuts is to run document.dispatchEvent(new KeyboardEvent("keydown", {key: "/", bubbles: true})); after the page loads. - You don't need Flask or Starlette to run a CORS-enabled localhost server for capturing JSON from another window - 19 lines of code against the Python standard library http.server package works just fine. - getComputedStyle(document.querySelector("navigation-search").shadowRoot.querySelector("textarea")) works to read dimensions from inside a Web Component's shadow DOM. - defaults write com.google.chrome.for.testing AppleShowScrollBars Always - Claude Fable knows how to apply all of the above. It's always interesting to pick up hints of what a model can and cannot do. I'm always confused at how many people equate using a coding agent to solve a problem with "learning nothing". If you pay attention to what it's doing you can learn so much!
  - saberience 4mo ago
    And Fable is still worse than Codex. I use both and the only thing (as always) that I will use Claude for is UI design. Opus 4.8 and now Fable are still both worse at actually getting the job done than the Codex model. Claude models write FAR too much code when it's not needed, they burn far too many tokens, when they are not needed, write un-necessary tests, write plans which are 5 pages longer than are needed, etc. etc. Have you actually compared code quality and plan quality versus Codex? It's demonstrably worse.
    - felixgallo 4mo ago
      In my experience writing about 50 programs with fable, opus, and GPT, fable is a significant step change better than opus which is significantly better than GPT. We must be doing different things.
- snowwrestler 4mo ago
  But Simon is not trying to get good at CSS debugging, Simon is trying to learn about AI systems and produce content about them. So giving the AI agent a trivial task to go crazy on is a feature, not a bug. For $12 implied cost, he got a front-page post on HN with 500 comments. What is that worth? :-)
  - xnorswap 4mo ago
    To most of us that's worth a ton, whereas he's probably had enough front-page posts that there's less value to him, although still likely more than $12 worth.
    - garblegarble 4mo ago
      >enough front-page posts that there's less value to him On the countrary I'd say it's probably even more important - without (amongst doing other "thought leader" things) getting on the HN front-page regularly an influencer's value to the industry disappears (not criticising him here)
      - simonw 4mo ago
        That's bad news for all of the other "AI influencers", off the top of my head I can't think of any with remotely my track record of hitting HN. (That's because they're all busy attracting millions of views on TikTok and YouTube, which are much more impactful channels than my dedication to blogging like it's 2005.)
        garblegarble 4mo ago
        That's what I meant by other thought leadership things - that's all covering different niches. For what it's worth, I think you do useful work and are a respectible influencer. I'd also say don't be down about your use of blogging - I'd say it makes you more valuable, there aren't that many decision-makers who are going to sit through a bunch of breathless YouTube videos... P.S. I hope you don't object to me using the term influencer, assumed you were on-board with it since in your post announcing your sponsorship you referenced Freeman & Forrest, "influencers on tap" / "building turnkey influencer marketing programs as a service".
- deleted 4mo ago
  [deleted]
- justinclift 4mo ago
  > By offloading this trivial task to the LLM, Simon has abandoned the opportunity to evaluate the abstraction [...] While by itself that would be true, Simon commonly blogs about things he's up to. That action provides the opportunity for evaluation, and additionally evaluation by a wider audience. So, it's not the same scenario as non-bloggers offloading a task... :)
biztos 4mo ago
They might also ask why a bunch of static CSS inside a bunch of JavaScript is hiding inside __init__.py[0] - hopefully before trying to fix some detail of the CSS. (I'm surprised to see it actually, since my own use of Claude has mostly yielded well-structured code. But I'm not doing proper vibe-coding, more like friendly Socratic arguing with another engineer who happens to be a robot.) [0] https://github.com/datasette/datasette-agent/blob/main/datasette_agent/__init__.py https://github.com/datasette/datasette-agent/blob/main/datas...
- simonw 4mo ago
  Thanks for the prod, I've extracted that script out into a separate static file: https://github.com/datasette/datasette-agent/commit/fa505b82491784049b64fd7539bb825368c92a21#diff-7eda59b4b0a4b3d1ad068190361a449e1f0e05b94578edb364b152ffe0e4b320 https://github.com/datasette/datasette-agent/commit/fa505b82... (It was in Python because there were a couple of URLs that needed to be dynamically constructed by the server, but those are output as a small window.datasetteAgentJumpConfig object instead now.)
  - Ummmdf 4mo ago
    [dead]
  - frumiousirc 4mo ago
    [flagged]
- byproxy 4mo ago
  > friendly Socratic arguing with another engineer who happens to be a robot Ha! Same! Still feels like the best way to go about it, really. I know the dream is to one day remove humans from the loop... but I'll enjoy the dialectic while it still seems the most productive!
  - vadansky 4mo ago
    Same, I like to call it rubber duck coding (now the duck talks back!) Edit: Now I want an LLM connected rubber duck with a speaker/microphone that sees your screen
flyingshelf 4mo ago
[dead]
simonw 4mo ago
You missed what I think is the most interesting question: why does the bug appear in Safari macOS but not in Firefox, Chrome, or WebKit running inside of Playwright? (Dozens of people in this thread implying that any web dev should have known to solve it with overflow-x: hidden and not one of them have addressed that browser difference yet.)
- fragmede 4mo ago
  people pay good money to not have their shit rendered via Playwright!
- hennell 4mo ago
  I think any web dev knows not to question browser differences if it can be fixed without opening that can of worms.
- zeroonetwothree 4mo ago
  Safari has some differences in default scroll behavior. I’ve seen similar bugs pop up many times.
elicash 4mo ago
I misread your comment at first and thought you were insulting Simon Willison, rather than calling Claude Fable a bad developer, and so I'm commenting here to clarify it in case others also misread it. That first sentence threw me off. Anyway, I'm glad he spent the $12 because this blog post was highly informative.
geysersam 4mo ago
This is the worst thing about current AI agents. They never ask questions. The prompt has to be pixel perfect and unambiguous or they'll happily run away doing something ridiculous.
andy_ppp 4mo ago
Yes I agree, the solution committed is horrible, but nobody cares any more. We have entered a very strange parallel universe where because AI can work things out it's easier to take solutions that are sub optimal and just churn out (potentially) buggy features.
- simonw 4mo ago
  I care. If you can loosely point me in the direction of a better solution I'll do the extra work.
  - andy_ppp 4mo ago
    Interesting... I downloaded dataset-agent and removed various different styles from the textarea (with an intention of providing a PR) including the overflow-x: hidden and I tried Safari and Chrome with both the global Mac setting of Always showing scrollbars on and off. It NEVER shows the scrollbar for me. Do you have an extension installed that is doing something weird to your textareas? Maybe I'm doing it wrong but I think for now overflow-x is fine if you are experiencing it and I am not! Let's all get on with our lives... I was probably a bit overzealous about caring all that much about a perfectly fine CSS fix.
    - simonw 4mo ago
      Amusingly I just had Claude vibe code up a new tool and it has exactly the same bug! Safari only, you have to expand the "Document context" area to see it. Here's that HTML file (frozen at the version with the bug): https://github.com/simonw/tools/blob/e7a23e8a1083ea99a5b3ef5a6133cbb4976df643/openai-webrtc.html https://github.com/simonw/tools/blob/e7a23e8a1083ea99a5b3ef5... It's hosted here, but I've added the overflow-x: hidden now so it's fixed: https://tools.simonwillison.net/openai-webrtc https://tools.simonwillison.net/openai-webrtc The bug only shows up if you increase your browser font size - at default size there's no scrollbar.
dreis_sw 4mo ago
Seems like this model delivers on what has already been scaling quite nicely, which is the length and complexity of the requested tasks, but isn't such a big improvement on what hasn't been scaling so far - common sense, discernment, good judgement.
- nlawalker 4mo ago
  > common sense, discernment, good judgement I feel like the whole point of all the experimentation with AI right now is determining whether any of these things actually matter to the end result, over various timeframes.
  - RealityVoid 4mo ago
    They matter.
    - pertymcpert 4mo ago
      Because?
      - deleted 4mo ago
        [deleted]
      - RealityVoid 4mo ago
        Because poor judgement leads to poor decisions.
        cyanydeez 4mo ago
        poor decisions are about context, direction and volition. All things LLMs will never have; sure AI might one day, but these systems are really good at solving complex problems with fantastical solutions while every force is just one hallucination away. simonw should spend more time trying to figure the sources of the information it used; that would be a wild ride, use the AI for all I care, we're all standing on the shoulders of giants but sourcing the giant as some mysterical thing.
Illniyar 4mo ago
I think Fable is predisposed to try and verify it's changes. Which is a very good thing. It takes a lot of prompts to get Opus to do what Fable does unprompted. That is exactly what I would want from a junior developer - make sure the bug exists, find a way to fix it, verify the bug is fixed. The problem, as was correctly identified in the blog post - is that instead of stopping and asking for elevated permission it relentlessly tries to find a hack on it's own. (An equivalent situation for a human developer would be needing some access to a third-party sandbox, and instead of asking a senior for credentials, tries to setup his own sandbox from scratch)
- AtNightWeCode 4mo ago
  No, the problem is mostly the incorrect prompt that sent fable into a rabbit hole resulting in an incorrect solution.
subygan 4mo ago
This is missing the point, simon is a fantastic developer. but to keep track of all the nuances of the frontend frameworks and browser implementation is a lot even for great people. it is really awesome that the final change was only a two line css change.
- AtNightWeCode 4mo ago
  But the fix is wrong as pointed out by the poster...
deleted 4mo ago
[deleted]
l33tman 4mo ago
[dead]
m463 4mo ago
Actually, it seems to me that it is just over-monetization of any impulse. I remember when you were billed by the minute for connecting to the online world. There were lots of incentives to keep the meter running. is this sort of like that?

gib444 4mo ago

The 'better' fixes are often for our (human) benefit. These messy fixes serve the AI companies' interests of creating messes that need even more tokens (money) later. Bad and self-serving developers also act the same, creating tech debt

piker 4mo ago

This is exactly right. By offloading this trivial task to the LLM, Simon has abandoned the opportunity to evaluate the abstraction with additional information and improve it. Instead, we let the agent spend $12 and make the fix while learning nothing.

discordance 4mo ago
I see it as a prioritization exercise. I know the above is a trivial example, but more generally, does the guy who wrote Datasette and Django want to wrangle front end and css, or do they want to work on something else?
- smartbit 4mo ago
  See above https://news.ycombinator.com/item?id=48498573#48502311 https://news.ycombinator.com/item?id=48498573#48502311
oulipo2 4mo ago
[flagged]
- beernet 4mo ago
  [flagged]
  - harperlee 4mo ago
    It was posted at 5am in New York... not sure that that was a US view, so the fact that the platform is US-owned doesn't seem so relevant, if there's a global audience. That being said, I do agree it is a legit thought (and moreso, completely on point in the subthread discussing downsides), and that it shouldn't be downvoted.
  - user43928 4mo ago
    While one can raise environmental concerns about the AI datacenter buildout, I don't think it is fair to say that it "ruins the planet". I don't think it is a good contribution to the discussion around Simon's LLM use to fix a CSS bug.
  - _heimdall 4mo ago
    That's an interesting choice as a source. It doesn't mention climate change or human impacts at all and describes El Niño as a naturally occurring event. > The El Nino is a phenomenon that occurs naturally
    - brickers 4mo ago
      El Niño is a naturally occurring event
    - dahart 4mo ago
      El Niño has been occurring naturally for more than 10,000 years. https://en.wikipedia.org/wiki/El_Ni%C3%B1o%E2%80%93Southern_Oscillation#History https://en.wikipedia.org/wiki/El_Ni%C3%B1o%E2%80%93Southern_...
jmmcd 4mo ago
People are missing that Willison is among the very best people we have in the role of (for lack of a good name): early access to frontier models, evaluate them in real scenarios, no wishful thinking, hype, or doom, communicate the possibilities. Yes he could have fixed this himself but then he would have learned nothing about the AI, and we wouldn't have read a fascinating and important article.
- risyachka 4mo ago
  >> he would have learned nothing about the AI there is absolutely zero value in spending time to learn about new models as in few months new model will be out and whatever you learned about the current one will be useless. Also with models getting better and better you have to know less and less to achieve same results.
  - simonw 4mo ago
    My experience has been the exact opposite. As the models get better you need to know more about their capabilities, because otherwise you risk prompting Claude Fable 5 like it's GPT-4o and complaining loudly about how it's all hype and nothing about these models is improving at all (yes, I do see people say that.) Getting the best results out of these models requires skill, experience, intuition, and domain expertise. There's always room for improving every one of those.
    - ViscountPenguin 4mo ago
      Eh, I've have the exact opposite experience. Way back before instruct models it was pretty difficult, but for the last couple of years I haven't needed anything more complex than the type of text that I might send in a detailed email to a colleague.
    - philipwhiuk 4mo ago
      Isn't the whole point of a better model that it should be better at understanding you than the previous one? So the same prompt should return a better answer. Prompting differently to the new model seems entirely backwards when trying to determine if the model has improved.
simonw 4mo ago
Things I learned from this: - Fable will do a whole lot more than you might expect in order to verify a fix. I learned that it's "relentlessly proactive". That's a good title for a blog entry! - You can take screenshots of a window in macOS using the "screencapture" CLI command, but you'll need the integer window ID first. - That windowID is accessible via "Quartz.CGWindowListCopyWindowInfo(Quartz.kCGWindowListOptionOnScreenOnly, Quartz.kCGNullWindowID)" using the pyobjc-framework-Quartz library, which installs cleanly via "uv run". - A neat trick for simulating keyboard shortcuts is to run document.dispatchEvent(new KeyboardEvent("keydown", {key: "/", bubbles: true})); after the page loads. - You don't need Flask or Starlette to run a CORS-enabled localhost server for capturing JSON from another window - 19 lines of code against the Python standard library http.server package works just fine. - getComputedStyle(document.querySelector("navigation-search").shadowRoot.querySelector("textarea")) works to read dimensions from inside a Web Component's shadow DOM. - defaults write com.google.chrome.for.testing AppleShowScrollBars Always - Claude Fable knows how to apply all of the above. It's always interesting to pick up hints of what a model can and cannot do. I'm always confused at how many people equate using a coding agent to solve a problem with "learning nothing". If you pay attention to what it's doing you can learn so much!
- saberience 4mo ago
  And Fable is still worse than Codex. I use both and the only thing (as always) that I will use Claude for is UI design. Opus 4.8 and now Fable are still both worse at actually getting the job done than the Codex model. Claude models write FAR too much code when it's not needed, they burn far too many tokens, when they are not needed, write un-necessary tests, write plans which are 5 pages longer than are needed, etc. etc. Have you actually compared code quality and plan quality versus Codex? It's demonstrably worse.
  - felixgallo 4mo ago
    In my experience writing about 50 programs with fable, opus, and GPT, fable is a significant step change better than opus which is significantly better than GPT. We must be doing different things.
snowwrestler 4mo ago
But Simon is not trying to get good at CSS debugging, Simon is trying to learn about AI systems and produce content about them. So giving the AI agent a trivial task to go crazy on is a feature, not a bug. For $12 implied cost, he got a front-page post on HN with 500 comments. What is that worth? :-)
- xnorswap 4mo ago
  To most of us that's worth a ton, whereas he's probably had enough front-page posts that there's less value to him, although still likely more than $12 worth.
  - garblegarble 4mo ago
    >enough front-page posts that there's less value to him On the countrary I'd say it's probably even more important - without (amongst doing other "thought leader" things) getting on the HN front-page regularly an influencer's value to the industry disappears (not criticising him here)
    - simonw 4mo ago
      That's bad news for all of the other "AI influencers", off the top of my head I can't think of any with remotely my track record of hitting HN. (That's because they're all busy attracting millions of views on TikTok and YouTube, which are much more impactful channels than my dedication to blogging like it's 2005.)
      - garblegarble 4mo ago
        That's what I meant by other thought leadership things - that's all covering different niches. For what it's worth, I think you do useful work and are a respectible influencer. I'd also say don't be down about your use of blogging - I'd say it makes you more valuable, there aren't that many decision-makers who are going to sit through a bunch of breathless YouTube videos... P.S. I hope you don't object to me using the term influencer, assumed you were on-board with it since in your post announcing your sponsorship you referenced Freeman & Forrest, "influencers on tap" / "building turnkey influencer marketing programs as a service".
deleted 4mo ago
[deleted]
justinclift 4mo ago
> By offloading this trivial task to the LLM, Simon has abandoned the opportunity to evaluate the abstraction [...] While by itself that would be true, Simon commonly blogs about things he's up to. That action provides the opportunity for evaluation, and additionally evaluation by a wider audience. So, it's not the same scenario as non-bloggers offloading a task... :)

biztos 4mo ago

They might also ask why a bunch of static CSS inside a bunch of JavaScript is hiding inside __init__.py[0] - hopefully before trying to fix some detail of the CSS. (I'm surprised to see it actually, since my own use of Claude has mostly yielded well-structured code. But I'm not doing proper vibe-coding, more like friendly Socratic arguing with another engineer who happens to be a robot.) [0] https://github.com/datasette/datasette-agent/blob/main/datasette_agent/__init__.py https://github.com/datasette/datasette-agent/blob/main/datas...

simonw 4mo ago
Thanks for the prod, I've extracted that script out into a separate static file: https://github.com/datasette/datasette-agent/commit/fa505b82491784049b64fd7539bb825368c92a21#diff-7eda59b4b0a4b3d1ad068190361a449e1f0e05b94578edb364b152ffe0e4b320 https://github.com/datasette/datasette-agent/commit/fa505b82... (It was in Python because there were a couple of URLs that needed to be dynamically constructed by the server, but those are output as a small window.datasetteAgentJumpConfig object instead now.)
- Ummmdf 4mo ago
  [dead]
- frumiousirc 4mo ago
  [flagged]
byproxy 4mo ago
> friendly Socratic arguing with another engineer who happens to be a robot Ha! Same! Still feels like the best way to go about it, really. I know the dream is to one day remove humans from the loop... but I'll enjoy the dialectic while it still seems the most productive!
- vadansky 4mo ago
  Same, I like to call it rubber duck coding (now the duck talks back!) Edit: Now I want an LLM connected rubber duck with a speaker/microphone that sees your screen

flyingshelf 4mo ago

[dead]

simonw 4mo ago

You missed what I think is the most interesting question: why does the bug appear in Safari macOS but not in Firefox, Chrome, or WebKit running inside of Playwright? (Dozens of people in this thread implying that any web dev should have known to solve it with overflow-x: hidden and not one of them have addressed that browser difference yet.)

fragmede 4mo ago
people pay good money to not have their shit rendered via Playwright!
hennell 4mo ago
I think any web dev knows not to question browser differences if it can be fixed without opening that can of worms.
zeroonetwothree 4mo ago
Safari has some differences in default scroll behavior. I’ve seen similar bugs pop up many times.

elicash 4mo ago

I misread your comment at first and thought you were insulting Simon Willison, rather than calling Claude Fable a bad developer, and so I'm commenting here to clarify it in case others also misread it. That first sentence threw me off. Anyway, I'm glad he spent the $12 because this blog post was highly informative.

geysersam 4mo ago

This is the worst thing about current AI agents. They never ask questions. The prompt has to be pixel perfect and unambiguous or they'll happily run away doing something ridiculous.

andy_ppp 4mo ago

Yes I agree, the solution committed is horrible, but nobody cares any more. We have entered a very strange parallel universe where because AI can work things out it's easier to take solutions that are sub optimal and just churn out (potentially) buggy features.

simonw 4mo ago
I care. If you can loosely point me in the direction of a better solution I'll do the extra work.
- andy_ppp 4mo ago
  Interesting... I downloaded dataset-agent and removed various different styles from the textarea (with an intention of providing a PR) including the overflow-x: hidden and I tried Safari and Chrome with both the global Mac setting of Always showing scrollbars on and off. It NEVER shows the scrollbar for me. Do you have an extension installed that is doing something weird to your textareas? Maybe I'm doing it wrong but I think for now overflow-x is fine if you are experiencing it and I am not! Let's all get on with our lives... I was probably a bit overzealous about caring all that much about a perfectly fine CSS fix.
  - simonw 4mo ago
    Amusingly I just had Claude vibe code up a new tool and it has exactly the same bug! Safari only, you have to expand the "Document context" area to see it. Here's that HTML file (frozen at the version with the bug): https://github.com/simonw/tools/blob/e7a23e8a1083ea99a5b3ef5a6133cbb4976df643/openai-webrtc.html https://github.com/simonw/tools/blob/e7a23e8a1083ea99a5b3ef5... It's hosted here, but I've added the overflow-x: hidden now so it's fixed: https://tools.simonwillison.net/openai-webrtc https://tools.simonwillison.net/openai-webrtc The bug only shows up if you increase your browser font size - at default size there's no scrollbar.

dreis_sw 4mo ago

Seems like this model delivers on what has already been scaling quite nicely, which is the length and complexity of the requested tasks, but isn't such a big improvement on what hasn't been scaling so far - common sense, discernment, good judgement.

nlawalker 4mo ago
> common sense, discernment, good judgement I feel like the whole point of all the experimentation with AI right now is determining whether any of these things actually matter to the end result, over various timeframes.
- RealityVoid 4mo ago
  They matter.
  - pertymcpert 4mo ago
    Because?
    - deleted 4mo ago
      [deleted]
    - RealityVoid 4mo ago
      Because poor judgement leads to poor decisions.
      - cyanydeez 4mo ago
        poor decisions are about context, direction and volition. All things LLMs will never have; sure AI might one day, but these systems are really good at solving complex problems with fantastical solutions while every force is just one hallucination away. simonw should spend more time trying to figure the sources of the information it used; that would be a wild ride, use the AI for all I care, we're all standing on the shoulders of giants but sourcing the giant as some mysterical thing.

Illniyar 4mo ago

I think Fable is predisposed to try and verify it's changes. Which is a very good thing. It takes a lot of prompts to get Opus to do what Fable does unprompted. That is exactly what I would want from a junior developer - make sure the bug exists, find a way to fix it, verify the bug is fixed. The problem, as was correctly identified in the blog post - is that instead of stopping and asking for elevated permission it relentlessly tries to find a hack on it's own. (An equivalent situation for a human developer would be needing some access to a third-party sandbox, and instead of asking a senior for credentials, tries to setup his own sandbox from scratch)

AtNightWeCode 4mo ago
No, the problem is mostly the incorrect prompt that sent fable into a rabbit hole resulting in an incorrect solution.

subygan 4mo ago

This is missing the point, simon is a fantastic developer. but to keep track of all the nuances of the frontend frameworks and browser implementation is a lot even for great people. it is really awesome that the final change was only a two line css change.

AtNightWeCode 4mo ago
But the fix is wrong as pointed out by the poster...

deleted 4mo ago

[deleted]

l33tman 4mo ago

[dead]

m463 4mo ago

Actually, it seems to me that it is just over-monetization of any impulse. I remember when you were billed by the minute for connecting to the online world. There were lots of incentives to keep the meter running. is this sort of like that?