3 ms·

Can you give an example because it has been my experience that they tend to give answers that sound right, but after closer inspection (or attempted compilation

by efnx 3y ago

Can you give an example because it has been my experience that they tend to give answers that sound right, but after closer inspection (or attempted compilation - or running) are in fact wrong.

baq 3y ago
Dogs are obviously (I hope) a joke, actually they’re word calculators. They have some knowledge embedded because that’s needed to be able to be a word calculator but getting it facts in the context window works much better, hence they’re good at summarizing etc.
LeifCarrotson 3y ago
They occasionally give answers that sound right but are in fact wrong. Their suitability for a particular task depends largely on your tolerance for that category of error. If you're generating fiction, it's irrelevant, if you're generating a newsletter that might be skimmed at a high level by disinterested people (or other LLMs) it's a low risk, if you're writing UI code it can be bad but will probably be corrected if it's bad enough, if you're writing critical code an insidious error can be disastrous.
brucethemoose2 3y ago
> or attempted compilation This specifically is a solvable problem with grammar constraints on the token output. You can give LLM no other option than syntactically correct C code, chess moves or whatever with the right grammar restriction. Its one of the (IMO) biggest revolutions in the LLM space that precisely no one uses.
deleted 3y ago
[deleted]
blitz_skull 3y ago
Right this moment I’m using it to iterate on a color gradient algorithm. It’s not perfect, but it’s outputting “close enough” stuff that I’m getting useful steps forward. Otherwise I’d be a couple hours into reading up on what the hell HSL is in color theory. It streamlines work that would otherwise require me to have extensive knowledge in an alternate domain. These tools are so much more useful if viewed as “coding buddies” who may get it wrong as much as you do, but they’re also skilled in several domains that you’re not. But bouncing ideas back and forth is crazy valuable.

by efnx 3y ago

Can you give an example because it has been my experience that they tend to give answers that sound right, but after closer inspection (or attempted compilation - or running) are in fact wrong.

baq 3y ago

Dogs are obviously (I hope) a joke, actually they’re word calculators. They have some knowledge embedded because that’s needed to be able to be a word calculator but getting it facts in the context window works much better, hence they’re good at summarizing etc.

LeifCarrotson 3y ago

They occasionally give answers that sound right but are in fact wrong. Their suitability for a particular task depends largely on your tolerance for that category of error. If you're generating fiction, it's irrelevant, if you're generating a newsletter that might be skimmed at a high level by disinterested people (or other LLMs) it's a low risk, if you're writing UI code it can be bad but will probably be corrected if it's bad enough, if you're writing critical code an insidious error can be disastrous.

brucethemoose2 3y ago

> or attempted compilation This specifically is a solvable problem with grammar constraints on the token output. You can give LLM no other option than syntactically correct C code, chess moves or whatever with the right grammar restriction. Its one of the (IMO) biggest revolutions in the LLM space that precisely no one uses.

deleted 3y ago

[deleted]

blitz_skull 3y ago

Right this moment I’m using it to iterate on a color gradient algorithm. It’s not perfect, but it’s outputting “close enough” stuff that I’m getting useful steps forward. Otherwise I’d be a couple hours into reading up on what the hell HSL is in color theory. It streamlines work that would otherwise require me to have extensive knowledge in an alternate domain. These tools are so much more useful if viewed as “coding buddies” who may get it wrong as much as you do, but they’re also skilled in several domains that you’re not. But bouncing ideas back and forth is crazy valuable.