7 ms·

Developing with GitHub Copilot Agent Mode and MCP

93 points by miltonlaxer 1y ago

luckystarr 1y ago
Playwright MCP is intriguing. I'll definitely give it a run today. Anybody got any tipps or gotchas?
- Kostarrr 1y ago
  If they didn't change it, Playwright uses the aria (accessibility) representation for their MCP agent. It strongly depends on the web page whether or not that yields good results. We at Octomind use a mix of augmented screenshots and page representation to guide the agent. If Playwright MCP doesnt work on you page, give our MCP a try. We have a free tier.
- never_inline 1y ago
  Can someone elucidate how using a full blown browser is improvisation over using say markitdown / pandoc / whatever? Given that most useful coding docs sites are static (made with sphinx or mkdocs or whatever)
skydhash 1y ago
And again, the most convoluted setup for development with an example that fails to demonstrate why you should adopt such practice. It’s like doing a GDB demo with an hello world program. Or doing Linux From Scratch to show how you can browse the web. The goal of software engineering is not to write code faster. Coding is itself a translation task (and a learning workflow, as you can’t keep everything in your head). What you want is the power of decision, and better decision can be made with better information. There’s nothing in the setup that helps with making decision. There are roughly six steps in software engineering, done sequentially and iteratively. Requirements gathering to shape the problem, Analysis to understand it, Design to come up with a solution, Coding to implement it, Testing to verify the solution, and Maintenance to keep the solution working. We have methods and tooling that help with each, giving us relevant information based on important parameters that we need to decide upon. LLMs are example generators. Give it a prompt and it will gives the answer that fits the conversation. It’s an echo chamber powered by a lossy version of the internet. Unlike my linting tool which will show me the error when there’s one and not when I tell it to. ADDENDUM It's like an ivory tower filled with yes-men and mirrors that always reply "you're the fairest of them all". My mind is already prone to lie to itself. What I need most is tooling that is not influenced by what I told it, or what others believe in. My browser is not influencing my note taking tool, telling it to note down the first two results it got from google. My editor is not telling the linter to sweep that error under a virtual rug. And QA does not care that I've implemented the most advanced abstraction if the software does not fit the specs.
- mvanbaak 1y ago
  I fear for the coming 2 to 3 generation of software engineers. Will they be able to handle problems if the AI is not available or is the source of the problem? Only time will tell.
  - mtkd 1y ago
    Same was said about dejanews, stackoverflow etc. and intellisense
    - alterom 1y ago
      Stack overflow didn't create a positive feedback loop where the solution to having to deal with an obscure, badly written, incomprehensible code base is creating an more incomprehensible sloppy code to glue it all together. Neither did intellisense. If anything, it encouraged structuring your code better so that intellisense would be useful. Intellisense does little for spaghetti code. And it was my #1 motivation to document the code in a uniform way, too. The most important impact of tools is that they change the way we think and see the world, and this shapes the world we create with these tools. When you hold a hammer, everything is a nail, as the saying goes. And when you hold a gun, you're no longer a mere human; you're a gunman. And the solution space for all sorts of problems starts looking very differently. The AI debate is not dissimilar to the gun debate. Yes, both guns and the AI are powerful tools that we have to deal with now that they've been invented. And people wielding these tools have an upper hand over those who don't. The point that people make in both debates that tends to get ignored by the proponents of these tools is that excessive use of the tools is exacerbating the very problem these tools are ostensibly solving. Giving guns to all schoolchildren won't solve the problem of high school shootings — it will undeniably make it worse. And giving the AI to all software developers won't solve the problem of bad, broken code that negatively impacts people who interact with it (as either users or developers). Finally, a note. Both the gun technology and the AI have been continuously improved since their invention. The progress is undeniable. Anyone who is thinking about guns in 1850 terms is making a mistake; the Maxim was a game changer. And we're not living in ChatGPT 2.0 times either. But with all the progress made, the solution space that either tool created hasn't been changing in nature. A problem that wasn't solveable with a flintlock musket or several remains intractable for an AK-74 or an M16. Improvements in either tech certainly did change the scale at which the tools were applied to resolve all sorts of problems. And the first half of the 20th century, to this day, provides most of the most brilliant, masterful examples of using guns at scale. What is also true is that the problems never went away. Nor did better guns made the lives of the common soldier any better. The work of people like nurse Nightingale did. And most of that work was that the solution to increasingly devastating battlefield casualties and dropping battlefield effectiveness wasn't giving every soldier a Maxim gun — it was better hygiene and living conditions. Washing hands. The Maxim gun was a game changer, but it wasn't a solution. The solution was getting out of the game with stupid prizes (like dying of cholera or typhoid fever). And it was an organizational issue, not a technological one. * * * * * To end on a good note, an observation for the AI doomers. Genocides have predated the guns by millenia, and more people have died by the machete and the bayonet than by any other weapon even in the 20th century. Perhaps the 21st too. Add disease and famine, and death by gun are a drop in the bucket. Guns aren't a solution to violence, but they're not, in themselves, a cause of it on a large enough scale. Mass production of guns made it possible to turn everyone into a soldier (and a target), but the absolute majority of people today have never seen war. And while guns, by design, are harmful — — they're also hella fun.
mohsen1 1y ago
I've had success using BrowserMCP https://browsermcp.io https://browsermcp.io It really feels magical when the AI agent can browse and click around to understand the problem at hand Also, sometimes an interactive command can stop agents from doing things. I wrote a small wrapper to always return so agents never stop from working https://github.com/mohsen1/agentshell https://github.com/mohsen1/agentshell
jonstewart 1y ago
Just yesterday I was reading a critique of MCP that specifically mentioned the GitHub MCP server as being harder to use (from model perspective) and requiring more tokens than having the agent execute git commands directly. I am surprised to see it listed here and also surprised to see two different web search servers and the time one. I would appreciate more detail from the author about the utility of each MCP server—overloading an agent with servers seems like it could be counterproductive.
deleted 1y ago
[deleted]
jpalomaki 1y ago
Not sure how things are with Copilot, but with Claude Code a good alternative for MCP is in some cases old fashioned command line tools. GitHub has gh, there's open source jira-cli, Cloudflare has wrangler and so on. No configuration needed, just mention on the agent doc that this kind of tool is available. Likely it will figure out the rest. And if you have more complicated needs, then you can combine the commands, add some jq magic, put to package.json and tell agent to use npm run to execute it. Can be faster than doing it via multiple MCP calls.
skatanski 1y ago
Really cool article. Personally I think the really cool bit about MCP is that you can very easily write your own server which can talk to the db or call various APIs. That server can run locally and be used by GitHub Copilot for answering questions and executing tasks. I also find it useful in a tight corporate environment where it’s more difficult to get a dedicated LLM API key. You can easily do POCa with what every dev has access to.
WhitneyLand 1y ago
What’s with Copilot “agent mode” anyway, how does it compare to using Claude Code or Gemini CLI?
- zihotki 1y ago
  I haven't used the latter two. The Claude Code experience is very well translatable to Copilot and they are similar in features. There were not many examples of Github Copilot advanced usage like described in the article when Agent mode was just released (around 2 months ago?). But I was able to fully utilize it using Claude Code prompts and examples. Copilot has a very direct advantage in my eyes - it has a plenty of models available: Sonnet 3.7, 4, GPT, Gemini, etc. That's something you won't get with the latter two.
deleted 1y ago
[deleted]
greatgib 1y ago
All of that to generate crappy code...
deleted 1y ago
[deleted]

93 points by miltonlaxer 1y ago

luckystarr 1y ago

Playwright MCP is intriguing. I'll definitely give it a run today. Anybody got any tipps or gotchas?

Kostarrr 1y ago
If they didn't change it, Playwright uses the aria (accessibility) representation for their MCP agent. It strongly depends on the web page whether or not that yields good results. We at Octomind use a mix of augmented screenshots and page representation to guide the agent. If Playwright MCP doesnt work on you page, give our MCP a try. We have a free tier.
never_inline 1y ago
Can someone elucidate how using a full blown browser is improvisation over using say markitdown / pandoc / whatever? Given that most useful coding docs sites are static (made with sphinx or mkdocs or whatever)

skydhash 1y ago

And again, the most convoluted setup for development with an example that fails to demonstrate why you should adopt such practice. It’s like doing a GDB demo with an hello world program. Or doing Linux From Scratch to show how you can browse the web. The goal of software engineering is not to write code faster. Coding is itself a translation task (and a learning workflow, as you can’t keep everything in your head). What you want is the power of decision, and better decision can be made with better information. There’s nothing in the setup that helps with making decision. There are roughly six steps in software engineering, done sequentially and iteratively. Requirements gathering to shape the problem, Analysis to understand it, Design to come up with a solution, Coding to implement it, Testing to verify the solution, and Maintenance to keep the solution working. We have methods and tooling that help with each, giving us relevant information based on important parameters that we need to decide upon. LLMs are example generators. Give it a prompt and it will gives the answer that fits the conversation. It’s an echo chamber powered by a lossy version of the internet. Unlike my linting tool which will show me the error when there’s one and not when I tell it to. ADDENDUM It's like an ivory tower filled with yes-men and mirrors that always reply "you're the fairest of them all". My mind is already prone to lie to itself. What I need most is tooling that is not influenced by what I told it, or what others believe in. My browser is not influencing my note taking tool, telling it to note down the first two results it got from google. My editor is not telling the linter to sweep that error under a virtual rug. And QA does not care that I've implemented the most advanced abstraction if the software does not fit the specs.

mvanbaak 1y ago
I fear for the coming 2 to 3 generation of software engineers. Will they be able to handle problems if the AI is not available or is the source of the problem? Only time will tell.
- mtkd 1y ago
  Same was said about dejanews, stackoverflow etc. and intellisense
  - alterom 1y ago
    Stack overflow didn't create a positive feedback loop where the solution to having to deal with an obscure, badly written, incomprehensible code base is creating an more incomprehensible sloppy code to glue it all together. Neither did intellisense. If anything, it encouraged structuring your code better so that intellisense would be useful. Intellisense does little for spaghetti code. And it was my #1 motivation to document the code in a uniform way, too. The most important impact of tools is that they change the way we think and see the world, and this shapes the world we create with these tools. When you hold a hammer, everything is a nail, as the saying goes. And when you hold a gun, you're no longer a mere human; you're a gunman. And the solution space for all sorts of problems starts looking very differently. The AI debate is not dissimilar to the gun debate. Yes, both guns and the AI are powerful tools that we have to deal with now that they've been invented. And people wielding these tools have an upper hand over those who don't. The point that people make in both debates that tends to get ignored by the proponents of these tools is that excessive use of the tools is exacerbating the very problem these tools are ostensibly solving. Giving guns to all schoolchildren won't solve the problem of high school shootings — it will undeniably make it worse. And giving the AI to all software developers won't solve the problem of bad, broken code that negatively impacts people who interact with it (as either users or developers). Finally, a note. Both the gun technology and the AI have been continuously improved since their invention. The progress is undeniable. Anyone who is thinking about guns in 1850 terms is making a mistake; the Maxim was a game changer. And we're not living in ChatGPT 2.0 times either. But with all the progress made, the solution space that either tool created hasn't been changing in nature. A problem that wasn't solveable with a flintlock musket or several remains intractable for an AK-74 or an M16. Improvements in either tech certainly did change the scale at which the tools were applied to resolve all sorts of problems. And the first half of the 20th century, to this day, provides most of the most brilliant, masterful examples of using guns at scale. What is also true is that the problems never went away. Nor did better guns made the lives of the common soldier any better. The work of people like nurse Nightingale did. And most of that work was that the solution to increasingly devastating battlefield casualties and dropping battlefield effectiveness wasn't giving every soldier a Maxim gun — it was better hygiene and living conditions. Washing hands. The Maxim gun was a game changer, but it wasn't a solution. The solution was getting out of the game with stupid prizes (like dying of cholera or typhoid fever). And it was an organizational issue, not a technological one. * * * * * To end on a good note, an observation for the AI doomers. Genocides have predated the guns by millenia, and more people have died by the machete and the bayonet than by any other weapon even in the 20th century. Perhaps the 21st too. Add disease and famine, and death by gun are a drop in the bucket. Guns aren't a solution to violence, but they're not, in themselves, a cause of it on a large enough scale. Mass production of guns made it possible to turn everyone into a soldier (and a target), but the absolute majority of people today have never seen war. And while guns, by design, are harmful — — they're also hella fun.

mohsen1 1y ago

I've had success using BrowserMCP https://browsermcp.io https://browsermcp.io It really feels magical when the AI agent can browse and click around to understand the problem at hand Also, sometimes an interactive command can stop agents from doing things. I wrote a small wrapper to always return so agents never stop from working https://github.com/mohsen1/agentshell https://github.com/mohsen1/agentshell

jonstewart 1y ago

Just yesterday I was reading a critique of MCP that specifically mentioned the GitHub MCP server as being harder to use (from model perspective) and requiring more tokens than having the agent execute git commands directly. I am surprised to see it listed here and also surprised to see two different web search servers and the time one. I would appreciate more detail from the author about the utility of each MCP server—overloading an agent with servers seems like it could be counterproductive.

deleted 1y ago

[deleted]

jpalomaki 1y ago

Not sure how things are with Copilot, but with Claude Code a good alternative for MCP is in some cases old fashioned command line tools. GitHub has gh, there's open source jira-cli, Cloudflare has wrangler and so on. No configuration needed, just mention on the agent doc that this kind of tool is available. Likely it will figure out the rest. And if you have more complicated needs, then you can combine the commands, add some jq magic, put to package.json and tell agent to use npm run to execute it. Can be faster than doing it via multiple MCP calls.

skatanski 1y ago

Really cool article. Personally I think the really cool bit about MCP is that you can very easily write your own server which can talk to the db or call various APIs. That server can run locally and be used by GitHub Copilot for answering questions and executing tasks. I also find it useful in a tight corporate environment where it’s more difficult to get a dedicated LLM API key. You can easily do POCa with what every dev has access to.

WhitneyLand 1y ago

What’s with Copilot “agent mode” anyway, how does it compare to using Claude Code or Gemini CLI?

zihotki 1y ago
I haven't used the latter two. The Claude Code experience is very well translatable to Copilot and they are similar in features. There were not many examples of Github Copilot advanced usage like described in the article when Agent mode was just released (around 2 months ago?). But I was able to fully utilize it using Claude Code prompts and examples. Copilot has a very direct advantage in my eyes - it has a plenty of models available: Sonnet 3.7, 4, GPT, Gemini, etc. That's something you won't get with the latter two.

deleted 1y ago

[deleted]

greatgib 1y ago

All of that to generate crappy code...

deleted 1y ago

[deleted]