4 ms·

I find this confusing: I can see the value in having an LLM assist you in developing a CI/CD workflow, but why would you want one involved in any continuous deg

I find this confusing: I can see the value in having an LLM assist you in developing a CI/CD workflow, but why would you want one involved in any continuous degree with your CI/CD? Perhaps it’s not as bad as that given that there’s a “compilation” phase, but the value add there isn’t super clear either (why would I check in both the markdown and the generated workflow; should I always regenerate from the markdown when I need changes, etc.). Given GitHub’s already lackluster reputation around security in GHA, I think I’d like to see them address some of GHA’s fundamental weaknesses before layering additional abstractions atop it.

zozbot234 9mo ago
> I find this confusing: I can see the value in having an LLM assist you in developing a CI/CD workflow, but why would you want one involved in any continuous degree with your CI/CD? The sensible case for this is for delivering human-facing project documentation, not actual code. (E.g. ask the AI agent to write its own "code review" report after looking at recent commits.) It's implemented using CI/CD solutions under the hood, but not real CI/CD.
- woodruffw 9mo ago
  Sorry, maybe I phrased my original comment poorly: I agree there's value in that kind of "self" code-review or other agent-driven workflow; I'm less clear on how that value is produced (performantly, reliably, etc.) by the architecture described on the site.
  - resquawk 9mo ago
    For Continuous Documentation examples, see https://github.github.io/gh-aw/blog/2026-01-13-meet-the-workflows-documentation/ https://github.github.io/gh-aw/blog/2026-01-13-meet-the-work...
mickdarling 9mo ago
I use an LLM behavior test to see if the semantic responses from LLMs using my MCP server match what I expect them to. This is beyond the regex tests, but to see if there's a semantic response that's appropriate. Sometimes the LLMs kick back an unusual response that technically is a no, but effectively is a yes. Different models can behave semantically different too. If I had a nice CI/CD workflow that was built into GitHub rather than rolling my own that I have running locally, that might just make it a little more automatic and a little easier.
blibble 9mo ago
> but why would you want one involved in any continuous degree with your CI/CD because helping you isn't the goal the goal is to generate revenue by consuming tokens and a never ending swarm of "AI" "agents" is a fantastic way to do that
wiether 9mo ago
I thought that it was to allow non-tech people to start making their own workflows/CI in a no/low-code way and compete against successful companies on this market. But the implementation is comically awful. Sure, you can "just write natural language" instructions and hope for the best. But they couldn't fully get away from their old demons and you still have to pay the YAML tax to set the necessary guardrails. I can't help but laugh at their example: https://github.com/github/gh-aw?tab=readme-ov-file#how-it-works https://github.com/github/gh-aw?tab=readme-ov-file#how-it-wo... They wrote 16 words in Markdown and... 19 in YAML. Because you can't trust the agent, you still have to write tons on gibberish YAML. I'm trying to understand it, but first you give permissions, here they only provide read permissions. And then give output permissions, which are actually write permissions on a smaller scope than the previous ones. Obviously they also absolve themselves from anything wrong that could happen by telling users to be careful. And they also suggest to setup an egress firewall to avoid the agents being too loose: https://github.com/github/gh-aw-firewall https://github.com/github/gh-aw-firewall Why setting-up an actual workflow engine on an infra managed by IT with actual security tooling when you can just stick together a few bits of YAML and Markdown on Github, right?
- resquawk 9mo ago
  The egress firewall is active by default, see https://github.github.io/gh-aw/introduction/architecture/ https://github.github.io/gh-aw/introduction/architecture/ We've fixed the example on the README to be a link, it should be clearer now what's going on.
ljm 9mo ago
I don't personally want any kind of workflow that spams my repo with gen AI refactorings or doc maintenance either. That is literally just creating overhead for me and it sounds like an excuse to shoehorn AI in to a workflow more than anything else.
- resquawk 9mo ago
  You are 100% in control.
resquawk 9mo ago
We've added an FAQ on determinism here: https://github.github.io/gh-aw/reference/faq/#determinism https://github.github.io/gh-aw/reference/faq/#determinism

zozbot234 9mo ago

> I find this confusing: I can see the value in having an LLM assist you in developing a CI/CD workflow, but why would you want one involved in any continuous degree with your CI/CD? The sensible case for this is for delivering human-facing project documentation, not actual code. (E.g. ask the AI agent to write its own "code review" report after looking at recent commits.) It's implemented using CI/CD solutions under the hood, but not real CI/CD.

woodruffw 9mo ago
Sorry, maybe I phrased my original comment poorly: I agree there's value in that kind of "self" code-review or other agent-driven workflow; I'm less clear on how that value is produced (performantly, reliably, etc.) by the architecture described on the site.
- resquawk 9mo ago
  For Continuous Documentation examples, see https://github.github.io/gh-aw/blog/2026-01-13-meet-the-workflows-documentation/ https://github.github.io/gh-aw/blog/2026-01-13-meet-the-work...

mickdarling 9mo ago

I use an LLM behavior test to see if the semantic responses from LLMs using my MCP server match what I expect them to. This is beyond the regex tests, but to see if there's a semantic response that's appropriate. Sometimes the LLMs kick back an unusual response that technically is a no, but effectively is a yes. Different models can behave semantically different too. If I had a nice CI/CD workflow that was built into GitHub rather than rolling my own that I have running locally, that might just make it a little more automatic and a little easier.

blibble 9mo ago

> but why would you want one involved in any continuous degree with your CI/CD because helping you isn't the goal the goal is to generate revenue by consuming tokens and a never ending swarm of "AI" "agents" is a fantastic way to do that

wiether 9mo ago

I thought that it was to allow non-tech people to start making their own workflows/CI in a no/low-code way and compete against successful companies on this market. But the implementation is comically awful. Sure, you can "just write natural language" instructions and hope for the best. But they couldn't fully get away from their old demons and you still have to pay the YAML tax to set the necessary guardrails. I can't help but laugh at their example: https://github.com/github/gh-aw?tab=readme-ov-file#how-it-works https://github.com/github/gh-aw?tab=readme-ov-file#how-it-wo... They wrote 16 words in Markdown and... 19 in YAML. Because you can't trust the agent, you still have to write tons on gibberish YAML. I'm trying to understand it, but first you give permissions, here they only provide read permissions. And then give output permissions, which are actually write permissions on a smaller scope than the previous ones. Obviously they also absolve themselves from anything wrong that could happen by telling users to be careful. And they also suggest to setup an egress firewall to avoid the agents being too loose: https://github.com/github/gh-aw-firewall https://github.com/github/gh-aw-firewall Why setting-up an actual workflow engine on an infra managed by IT with actual security tooling when you can just stick together a few bits of YAML and Markdown on Github, right?

resquawk 9mo ago
The egress firewall is active by default, see https://github.github.io/gh-aw/introduction/architecture/ https://github.github.io/gh-aw/introduction/architecture/ We've fixed the example on the README to be a link, it should be clearer now what's going on.

ljm 9mo ago

I don't personally want any kind of workflow that spams my repo with gen AI refactorings or doc maintenance either. That is literally just creating overhead for me and it sounds like an excuse to shoehorn AI in to a workflow more than anything else.

resquawk 9mo ago
You are 100% in control.

resquawk 9mo ago

We've added an FAQ on determinism here: https://github.github.io/gh-aw/reference/faq/#determinism https://github.github.io/gh-aw/reference/faq/#determinism