3 ms·

> The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide, and how to define the success cri

> The job then becomes identifying those problems and figuring out how to configure a sandbox for them, what tools to provide, and how to define the success criteria for the model. Your test case seems like a quintessential example where you're missing that last step. Since it is unlikely that you understand the math behind fractals or x86 assembly (apologies if I'm wrong on this), your only means for verifying the accuracy of your solution is a superficial visual inspection, e.g. "Does it look like the Mandelbrot series?" Ideally, your evaluation criteria would be expressed as a continuous function, but at the very least, it should take the form of a sufficiently diverse quantifiable set of discrete inputs and their expected outputs.

simonw 1y ago
That's exactly why I like using Mandelbrot as a demo: it's perfect for "superficial visual inspection". With a bunch more work I could likely have got a vision LLM to do that visual inspection for me in the assembly example, but having a human in the loop for that was much more productive.
shepherdjerred 1y ago
Are fractals or x86 assembly representative of most dev work?
- nartho 1y ago
  I think it's irrelevant. The point they are trying to make is anytime you ask a LLM for something that's outside of your area of expertise you have very little to no way to insure it is correct.
  - diggan 1y ago
    > anytime you ask a LLM for something that's outside of your area of expertise you have very little to no way to insure it is correct. I regularly use LLMs to code specific functions I don't necessarily understand the internals of. Most of the time I do that, it's something math-heavy for a game. Just like any function, I put it under automated and manual tests. Still, I review and try to gain some intuition about what is happening, but it is still very far of my area of expertise, yet I can be sure it works as I expect it to.
  - shepherdjerred 1y ago
    I think you're vastly overestimating the amount of knowledge the average developer has in their "area of expertise" I'm not saying that's a good thing, just that LLMs are no worse than the bottom 50% of devs.

simonw 1y ago

That's exactly why I like using Mandelbrot as a demo: it's perfect for "superficial visual inspection". With a bunch more work I could likely have got a vision LLM to do that visual inspection for me in the assembly example, but having a human in the loop for that was much more productive.

shepherdjerred 1y ago

Are fractals or x86 assembly representative of most dev work?

nartho 1y ago
I think it's irrelevant. The point they are trying to make is anytime you ask a LLM for something that's outside of your area of expertise you have very little to no way to insure it is correct.
- diggan 1y ago
  > anytime you ask a LLM for something that's outside of your area of expertise you have very little to no way to insure it is correct. I regularly use LLMs to code specific functions I don't necessarily understand the internals of. Most of the time I do that, it's something math-heavy for a game. Just like any function, I put it under automated and manual tests. Still, I review and try to gain some intuition about what is happening, but it is still very far of my area of expertise, yet I can be sure it works as I expect it to.
- shepherdjerred 1y ago
  I think you're vastly overestimating the amount of knowledge the average developer has in their "area of expertise" I'm not saying that's a good thing, just that LLMs are no worse than the bottom 50% of devs.