4 ms·
Has anyone hooked this up to a unit test system, like LLMtries = [] while(!testPassed) { - get new LLM try (w/ LLMtries history, and test results)
by yawnxyz 4y ago
Has anyone hooked this up to a unit test system, like
LLMtries = []
while(!testPassed) {
- get new LLM try (w/ LLMtries history, and test results)
- run/eval the try
- run the test
}
and kind of see how long it takes to generate the code that works? If it ever ends, the last LLMtries is the one that worked.
I haven't done this because I see this burning through lots of credits. However, if this thing costs $5k/year but is better than hiring a $50k a year engineer (or consultant)... I'd use it.
- blowski 4y agoMost engineering money is spent defining the test cases, and that doesn’t change here. It’s just that many organisations define test cases by first running something in production and then debugging it.
- LoganDark 4y ago> debugging it You mean putting its current behavior into the tests verbatim? :)
- sharemywin 4y agojust add if tried x tries and still doesn't work ask for help. and you just created a junior dev.
- deleted 4y ago[deleted]