6 ms·

Asking here because it seems related: I'm trying to use cursor to work on a webapp. It gets frustrating because vanilla Cursor is "coding blind" and can't actua

Asking here because it seems related: I'm trying to use cursor to work on a webapp. It gets frustrating because vanilla Cursor is "coding blind" and can't actually see the result of what it is doing, and whether or not it works. I ask it to fix something. It claims to know what the problem is, changes the code, and then claims it's fixed. I open app, and it's still broken. I have to continuously and way to often repeatedly tell it what it broken. Now, supposing I'm "vibe coding" and don't really care about the obvious fact that the AI doesn't actually know what it is doing, it's still frustrating that I have to be in the loop just to provide very basic information like that. Are there any agentic coding setups that allow the agent to interact with the app it's working on to check if it actually works?

tomashubelbauer 1y ago
Look into the Playwright MCP server, it allows coding agents to scrutinize the results of their work in the web browser. There is also an MCP server for the Chrome DevTools protocol AFAIK but I haven't tried it.
- artpar 1y ago
  I don't know if plywright works without chrome in debug mode, but I tried the MCP for chrome devtools and it requires chrome to be started in debugging mode and that basically means you cant log into a lot of sites (especially google) since it will block you with an "Unsafe" message. Works pretty well if you owe the target website.
- kevinsync 1y ago
  I was in the same boat on a side project (Electron, Claude Code) -- I considered Playwright but ended up building a simple, focused API instead that allows Claude to connect to the app to inspect logs (main console + browser console), query internal app data + state, and execute arbitrary JS. It's sped up debugging a lot since I can just give it instructions like "found a bug that does XYZ, I think it's a problem with functionABC(); connect to app, click these four buttons in this order, examine the internal state, then trace through the code to figure out what's going wrong and present a solution" I was pretty resistant at first of delegating debugging blindly like that, but it's made the workflow pretty smooth to where I can occasionally just open the app, run through it as a human user and take notes on bugs and flow issues that I find, log them with steps to reproduce, then give Claude a list of bugs to noodle on while I'm focusing on stuff LLMs are terrible at (design, UI, frontend work, etc)
shardullavekar 1y ago
a built-in mcp server that takes a look at what's broken and communicates with cursor is on our roadmap. Join discord and we will keep you posted there.
artpar 1y ago
So actually I have this setup (of a bridge server) which I use for agent4 itself (so claude code can talk to agent4), It makes a lot of sense to publish that bridge as well in the MCP form.
JimDabell 1y ago
You can use things like Browser Use and Playwright to hook things like that up, but you’re right, this is a very underdeveloped area. Armin Ronacher has a talk that covers some of this, such as unifying console.log, server logs, SQL, etc. to feed back to the LLM. https://www.youtube.com/watch?v=nfOVgz_omlU https://www.youtube.com/watch?v=nfOVgz_omlU
- wahnfrieden 1y ago
  This is the way. You can also feed screenshots back to it.
  - dcchuck 1y ago
    My experience with the off the shelf MCP tools is they still fall short of giving Claude or Codex a screenshot. I'm interested in whether or not others agree.
    - wahnfrieden 1y ago
      MCP or CLI (called by agent) can also be used to generate the screenshot input loop
      - dcchuck 1y ago
        Yes, but image size restrictions bring more failure than success in my experience. Thank you for your comment.
xnx 1y ago
Gemini CLI Chrome devtools MCP addresses this: https://developer.chrome.com/blog/chrome-devtools-mcp https://developer.chrome.com/blog/chrome-devtools-mcp
- hatmanstack 1y ago
  Jump the line and just install it. who needs to read stuff. https://github.com/ChromeDevTools/chrome-devtools-mcp?tab=readme-ov-file https://github.com/ChromeDevTools/chrome-devtools-mcp?tab=re...

tomashubelbauer 1y ago

Look into the Playwright MCP server, it allows coding agents to scrutinize the results of their work in the web browser. There is also an MCP server for the Chrome DevTools protocol AFAIK but I haven't tried it.

artpar 1y ago
I don't know if plywright works without chrome in debug mode, but I tried the MCP for chrome devtools and it requires chrome to be started in debugging mode and that basically means you cant log into a lot of sites (especially google) since it will block you with an "Unsafe" message. Works pretty well if you owe the target website.
kevinsync 1y ago
I was in the same boat on a side project (Electron, Claude Code) -- I considered Playwright but ended up building a simple, focused API instead that allows Claude to connect to the app to inspect logs (main console + browser console), query internal app data + state, and execute arbitrary JS. It's sped up debugging a lot since I can just give it instructions like "found a bug that does XYZ, I think it's a problem with functionABC(); connect to app, click these four buttons in this order, examine the internal state, then trace through the code to figure out what's going wrong and present a solution" I was pretty resistant at first of delegating debugging blindly like that, but it's made the workflow pretty smooth to where I can occasionally just open the app, run through it as a human user and take notes on bugs and flow issues that I find, log them with steps to reproduce, then give Claude a list of bugs to noodle on while I'm focusing on stuff LLMs are terrible at (design, UI, frontend work, etc)

shardullavekar 1y ago

a built-in mcp server that takes a look at what's broken and communicates with cursor is on our roadmap. Join discord and we will keep you posted there.

artpar 1y ago

So actually I have this setup (of a bridge server) which I use for agent4 itself (so claude code can talk to agent4), It makes a lot of sense to publish that bridge as well in the MCP form.

JimDabell 1y ago

You can use things like Browser Use and Playwright to hook things like that up, but you’re right, this is a very underdeveloped area. Armin Ronacher has a talk that covers some of this, such as unifying console.log, server logs, SQL, etc. to feed back to the LLM. https://www.youtube.com/watch?v=nfOVgz_omlU https://www.youtube.com/watch?v=nfOVgz_omlU

wahnfrieden 1y ago
This is the way. You can also feed screenshots back to it.
- dcchuck 1y ago
  My experience with the off the shelf MCP tools is they still fall short of giving Claude or Codex a screenshot. I'm interested in whether or not others agree.
  - wahnfrieden 1y ago
    MCP or CLI (called by agent) can also be used to generate the screenshot input loop
    - dcchuck 1y ago
      Yes, but image size restrictions bring more failure than success in my experience. Thank you for your comment.

xnx 1y ago

Gemini CLI Chrome devtools MCP addresses this: https://developer.chrome.com/blog/chrome-devtools-mcp https://developer.chrome.com/blog/chrome-devtools-mcp

hatmanstack 1y ago
Jump the line and just install it. who needs to read stuff. https://github.com/ChromeDevTools/chrome-devtools-mcp?tab=readme-ov-file https://github.com/ChromeDevTools/chrome-devtools-mcp?tab=re...