2 ms·

I didn’t quite understand why YAML is better than Markdown for such specifications. If the specification is written in such a strict format as YAML, I would ex

I didn’t quite understand why YAML is better than Markdown for such specifications. If the specification is written in such a strict format as YAML, I would expect it to be executable, something like this https://blog.fooqux.com/blog/executable-specification/ https://blog.fooqux.com/blog/executable-specification/ But as far as I understood, for acai that is not the case.

csomar 5mo ago
It's not. And LLMs don't do well with YAML either. I've had the agent/model struggle with `sed` trying to count how many spaces are in there multiple times to get the file to pass. It's the worst format you can use for LLMs.
brendanmc6 5mo ago
Here is how I got from Markdown to YAML in a few easy steps: 1. Start with unmaintainable prompt. Add auth to the admin route. It must only be accessible to team members. It must not be accessible to team members with the `quarantine` role. Unauthorized users should be redirected to the /login page, with query param `?error=unauthorized`. Requires fresh session, no older than 10 minutes. 2. Put it in a structured .md spec of some kind # Feat: Admin Route ## Auth requirements * [ ] Accessible to team members. * [ ] Not be accessible to members with `quarantine` role. * [ ] Redirects unauthorized users to /login * [ ] Redirect path includes `?error=unauthorized` * [ ] Requires fresh session, no older than 10 minutes. But how do you cross reference the requirements? Give them stable IDs? # Feat: Admin Route ## Requirements AUTH.1: Accessible to team members. AUTH.2: Not be accessible to members with `quarantine` role. AUTH.3: Redirects unauthorized users to /login. AUTH.3.1: Redirect path includes `?error=unauthorized`. AUTH.4: Requires fresh session, no older than 10 minutes. Might as well clean it up and make it machine readable at that point? feature: name: admin-route requirements: AUTH: 1: Accessible to team members. 2: Not be accessible to members with `quarantine` role. 3: Redirects unauthorized users to /login. 3-1: Redirect path includes `?error=unauthorized`. 4: Requires fresh session, no older than 10 minutes. ```
- dotneter 5mo ago
  > Might as well clean it up and make it machine readable at that point? For me this looks pretty machine readable. ## Requirements AUTH.1: Accessible to team members. AUTH.2: Not be accessible to members with `quarantine` role. AUTH.3: Redirects unauthorized users to /login. AUTH.3.1: Redirect path includes `?error=unauthorized`. AUTH.4: Requires fresh session, no older than 10 minutes. Yes, yaml is more "parseble". But I still need to use some format conventions inside yaml. So I could just use them inside markdown as well.
- physicles 5mo ago
  As someone who’s an experienced coder but sorta new to LLMs, I’ve learned that my intuition about what is and isn’t machine-readable for an LLM isn’t well-tuned. Better to do the benchmarking so you can be sure you’re not fooling yourself about whether use of YAML is warranted.

csomar 5mo ago

It's not. And LLMs don't do well with YAML either. I've had the agent/model struggle with `sed` trying to count how many spaces are in there multiple times to get the file to pass. It's the worst format you can use for LLMs.

brendanmc6 5mo ago

Here is how I got from Markdown to YAML in a few easy steps: 1. Start with unmaintainable prompt. Add auth to the admin route. It must only be accessible to team members. It must not be accessible to team members with the `quarantine` role. Unauthorized users should be redirected to the /login page, with query param `?error=unauthorized`. Requires fresh session, no older than 10 minutes. 2. Put it in a structured .md spec of some kind # Feat: Admin Route ## Auth requirements * [ ] Accessible to team members. * [ ] Not be accessible to members with `quarantine` role. * [ ] Redirects unauthorized users to /login * [ ] Redirect path includes `?error=unauthorized` * [ ] Requires fresh session, no older than 10 minutes. But how do you cross reference the requirements? Give them stable IDs? # Feat: Admin Route ## Requirements AUTH.1: Accessible to team members. AUTH.2: Not be accessible to members with `quarantine` role. AUTH.3: Redirects unauthorized users to /login. AUTH.3.1: Redirect path includes `?error=unauthorized`. AUTH.4: Requires fresh session, no older than 10 minutes. Might as well clean it up and make it machine readable at that point? feature: name: admin-route requirements: AUTH: 1: Accessible to team members. 2: Not be accessible to members with `quarantine` role. 3: Redirects unauthorized users to /login. 3-1: Redirect path includes `?error=unauthorized`. 4: Requires fresh session, no older than 10 minutes. ```

dotneter 5mo ago
> Might as well clean it up and make it machine readable at that point? For me this looks pretty machine readable. ## Requirements AUTH.1: Accessible to team members. AUTH.2: Not be accessible to members with `quarantine` role. AUTH.3: Redirects unauthorized users to /login. AUTH.3.1: Redirect path includes `?error=unauthorized`. AUTH.4: Requires fresh session, no older than 10 minutes. Yes, yaml is more "parseble". But I still need to use some format conventions inside yaml. So I could just use them inside markdown as well.
physicles 5mo ago
As someone who’s an experienced coder but sorta new to LLMs, I’ve learned that my intuition about what is and isn’t machine-readable for an LLM isn’t well-tuned. Better to do the benchmarking so you can be sure you’re not fooling yourself about whether use of YAML is warranted.