5 ms·

Yes, but honestly what's the best source when reporting about a person? Their personal website no? I think it's a hard problem and I feel there are a lot of tr

by zurfer 8mo ago

Yes, but honestly what's the best source when reporting about a person? Their personal website no? I think it's a hard problem and I feel there are a lot of trade-offs here. It's not as simple as saying chatgpt is stupid or the author shouldn't be surprised.

kulahan 8mo ago
The problem isn’t that it pulled the data from his personal site, it’s that it simply accepted his information which was completely false. It’s not a hard problem to solve at this time. “Oh, there’s exactly zero corroborating sources on this. I’ll ignore it.”
- moebrowne 8mo ago
  Verifying that something is 'true' requires more than corroborating sources. Making a second blog post on another domain is trivial, then a third and a forth.
  - kulahan 8mo ago
    I’m not going to write out the entire logic train for an LLM to determine whether or not one of the billion documents scanned that day is new. Of course you’ll need more than one simple “does anyone on the internet anywhere also say this” check. It’s obvious to everyone that I did not mean this one thing would somehow be a bulletproof, complete method of determining if something is true or not. It’d just an incredibly strong signal of inauthenticity. Come on man.
fatherwavelet 8mo ago
To me it is like steering a car into the ditch and then posting how the car went into a ditch. You don't have to drive that much to figure out that what is impressive is keeping the car on the road and then traveling further or faster than what you could do by walking. For that though you actually have to have a destination in mind and not just spin the wheels. Post pointless metrics on how fast the wheels spin for your blog no one reads in the vague hope of some hyper Warhol 15 milliseconds of "fame". The models for me are just making the output of the average person an insufferable bore.

by zurfer 8mo ago

kulahan 8mo ago

The problem isn’t that it pulled the data from his personal site, it’s that it simply accepted his information which was completely false. It’s not a hard problem to solve at this time. “Oh, there’s exactly zero corroborating sources on this. I’ll ignore it.”

moebrowne 8mo ago
Verifying that something is 'true' requires more than corroborating sources. Making a second blog post on another domain is trivial, then a third and a forth.
- kulahan 8mo ago
  I’m not going to write out the entire logic train for an LLM to determine whether or not one of the billion documents scanned that day is new. Of course you’ll need more than one simple “does anyone on the internet anywhere also say this” check. It’s obvious to everyone that I did not mean this one thing would somehow be a bulletproof, complete method of determining if something is true or not. It’d just an incredibly strong signal of inauthenticity. Come on man.

fatherwavelet 8mo ago

To me it is like steering a car into the ditch and then posting how the car went into a ditch. You don't have to drive that much to figure out that what is impressive is keeping the car on the road and then traveling further or faster than what you could do by walking. For that though you actually have to have a destination in mind and not just spin the wheels. Post pointless metrics on how fast the wheels spin for your blog no one reads in the vague hope of some hyper Warhol 15 milliseconds of "fame". The models for me are just making the output of the average person an insufferable bore.