3 ms·

My point is that to create a specification you need to use a formal language of some kind. In this example they created a new yaml based specification language.

My point is that to create a specification you need to use a formal language of some kind. In this example they created a new yaml based specification language. Why do that vs use a well documented existing formal language the LLM knows well like Python. The translation is either yaml -> new language or Python -> new language. The translation is happening in both cases. The advantage I can think of is it would might be more human readable but Python is damn close to pseudocode. It’ll likely always be a bit annoying to write because it has to be a formal language.

simonw 9mo ago
There's no translation from YAML to a different language. The YAML describes the tests - like this file here: https://github.com/dbreunig/whenwords/blob/main/tests.yaml https://github.com/dbreunig/whenwords/blob/main/tests.yaml Snippet: - name: "5 hours ago" input: { timestamp: 1704049200, reference: 1704067200 } output: "5 hours ago" - name: "21 hours ago" input: { timestamp: 1703991600, reference: 1704067200 } output: "21 hours ago" When told "use red/green TDD to write code for this in Ruby", a coding agent like Claude Code will write a test harness in Ruby that loops through all of those YAML tests, run it and watch it fail, then write just enough Ruby that the tests pass.
- roxolotl 9mo ago
  Yea I guess we're having a definitional disagreement here. To be clear I think this is a good idea and the work you've done using tests from projects to have agents translate libraries is awesome. But to me clearly that YAML snippet you provided is a specification which needs to be translated to Ruby as much as Python would. If the equivalent Python is: def test_timeago_5_hours_ago(self): self.assertEqual(timeago(1704049200, 1704067200)), "5 hours ago") def test_timeago_21_hours_ago(self): self.assertEqual(timeago(1703991600, 1704067200)), "21 hours ago") The YAML is no more clear than the Python, nor closer to Ruby. Honestly I think it's less clear as a human reading it because it's hard to tell which function is being tested in context of a specific test case. I guess it's possible Claude is better at working with the YAML than the Python but that would be a coincidence I think.

simonw 9mo ago

There's no translation from YAML to a different language. The YAML describes the tests - like this file here: https://github.com/dbreunig/whenwords/blob/main/tests.yaml https://github.com/dbreunig/whenwords/blob/main/tests.yaml Snippet: - name: "5 hours ago" input: { timestamp: 1704049200, reference: 1704067200 } output: "5 hours ago" - name: "21 hours ago" input: { timestamp: 1703991600, reference: 1704067200 } output: "21 hours ago" When told "use red/green TDD to write code for this in Ruby", a coding agent like Claude Code will write a test harness in Ruby that loops through all of those YAML tests, run it and watch it fail, then write just enough Ruby that the tests pass.

roxolotl 9mo ago
Yea I guess we're having a definitional disagreement here. To be clear I think this is a good idea and the work you've done using tests from projects to have agents translate libraries is awesome. But to me clearly that YAML snippet you provided is a specification which needs to be translated to Ruby as much as Python would. If the equivalent Python is: def test_timeago_5_hours_ago(self): self.assertEqual(timeago(1704049200, 1704067200)), "5 hours ago") def test_timeago_21_hours_ago(self): self.assertEqual(timeago(1703991600, 1704067200)), "21 hours ago") The YAML is no more clear than the Python, nor closer to Ruby. Honestly I think it's less clear as a human reading it because it's hard to tell which function is being tested in context of a specific test case. I guess it's possible Claude is better at working with the YAML than the Python but that would be a coincidence I think.