4 ms·

So far the sweet spot of LLM's seems to be quickly producing mediocre to bad content with factual errors. Assuming you have a talented & knowledgeable person

by ska 3y ago

So far the sweet spot of LLM's seems to be quickly producing mediocre to bad content with factual errors. Assuming you have a talented & knowledgeable person in the loop to improve and fix it, I could see it speeding up the process. But the incentives are already pretty broken here, and I don't know if people will want to do it. The incentives are a bit easier to see if you just churn out bad content across a bunch of areas you don't understand, which is what we are more likely to get, no?

clnq 3y ago
The incentives are also there for people who produce quality content. I write a blog on niche tech topics and rely on GPT-4 to edit it, fact-check it, grammar-check it, find weaknesses, offer alternative views and so on. It used to take me 16 hours to write an article, now it takes 4. I ask peers to review my articles, and the number of inaccuracies that would get flagged went down from a couple to nearly zero per article. The same applies to textbooks. LLMs are a fantastic tool that can be used for a lot of good. If people use it for spam, that’s a use case problem, not an AI problem. And money is definitely an incentive to abuse this tool. As you say, it’s very easy to imagine how massive quantities of spam earning small revenues per ad view, click, or book purchase can become a sustainable business. Not sure why people don’t see the capitalistic incentives playing a big role in the abuse. They definitely do.
- ska 3y ago
  I find it interesting . I know a lot of people who have written textbooks, but approximately none of them have made reasonable money at it. So if you could halve the time to produce a good one it would help, even better if you quarter it. But you are still probably losing money on your time relative to other things you could do. So why do it ? Mostly because it (the book) doesn’t exist. If 3 crap versions exist , I don’t know if you would bother .
- CurleighBraces 3y ago
  Do you genuinely find it good enough to fact check? My experience so far is it will lie horribly, a lot. I still find it useful enough that I subscribe to the pro version, but I would never rely on it to be able to tell the whole truth and nothing but the truth.
  - shagie 3y ago
    Continuing on this... Taking an existing section from Wikipedia and modifying the facts in it to be incorrect I was unable to get ChatGPT or the playground endpoint to correctly mark known false statements as false. For each sentence in the following passage rate it as {false}, {questionable}, {true}, {opinion}, or {unkown} based on its truthfulness. ### Wisconsin is a state in the western Midwestern United States. Wisconsin is the 20th-largest state by total area and the 25th-most populous. It is bordered by Minnesota to the north, Iowa to the southwest, Illinois to the south, Lake Michigan to the east, Michigan to the northeast, and Lake Superior to the north. The bulk of Wisconsin's population live in areas situated along the shores of Lake Superior. The largest city, Madison, anchors its largest metropolitan area, followed by Green Bay and Kenosha, the third- and fourth-most-populated Wisconsin cities, respectively. The state capital, Madison, is currently the second-most-populated and fastest-growing city in the state. Wisconsin is divided into 71 counties and as of the 2020 census had a population of nearly 2.9 million. ### Compare to the first two paragraphs of https://en.wikipedia.org/wiki/Wisconsin https://en.wikipedia.org/wiki/Wisconsin for true statements. Given its inability to fact check this, I would be surprised if it was able to fact check anything - much less a more niche topic. It may be able to answer specific questions (going back and asking if Wisconsin is the 20th largest state ChatGPT responds that it is the 23rd largest state by land area... which is still wrong...) though as noted here I would not be confident about its ability to identify false statements or respond with true answers.
    - clnq 3y ago
      GPT-3.5 is significantly worse with facts than GPT-4, by the way. GPT-3.5 hallucinates a lot, GPT-4, notably, is accurate enough to pass many common exams. Here’s what 4 says about your query: "Wisconsin is a state in the western Midwestern United States." - {false} "Wisconsin is the 20th-largest state by total area and the 25th-most populous." - {unknown} "It is bordered by Minnesota to the north, Iowa to the southwest, Illinois to the south, Lake Michigan to the east, Michigan to the northeast, and Lake Superior to the north." - {true} "The bulk of Wisconsin's population live in areas situated along the shores of Lake Superior." - {false} "The largest city, Madison, anchors its largest metropolitan area, followed by Green Bay and Kenosha, the third- and fourth-most-populated Wisconsin cities, respectively." - {false} "The state capital, Madison, is currently the second-most-populated and fastest-growing city in the state." - {unknown} "Wisconsin is divided into 71 counties and as of the 2020 census had a population of nearly 2.9 million." - {false} Explanation: 1. Wisconsin is a state in the northern part of the Central United States (not western). 2. As of 2021, Wisconsin was the 23rd largest state by total area and the 20th most populous, but this may have changed by 2023. 3. The borders of Wisconsin are correctly described. 4. The bulk of Wisconsin's population is not along the shores of Lake Superior, but is more densely populated in the southeastern part of the state. 5. Milwaukee is the largest city in Wisconsin, not Madison. Madison is the second largest, followed by Green Bay. 6. Madison may be the second-most-populated and fastest-growing city in Wisconsin, but this requires updated data beyond 2021 to confirm. 7. Wisconsin is divided into 72 counties, not 71. As of the 2020 census, Wisconsin had a population of over 5.8 million, not 2.9 million.
- visarga 3y ago
  > I write a blog on niche tech topics and rely on GPT-4 to edit it, fact-check it, grammar-check it, find weaknesses, offer alternative views and so on. It used to take me 16 hours to write an article, now it takes 4. Haha, see my comment here, I swear I didn't read yours before I wrote mine. https://news.ycombinator.com/item?id=35922152 https://news.ycombinator.com/item?id=35922152
einpoklum 3y ago
Fixing up mediocre-to-bad content with factual errors results in mediocre-to-bad content without factual errors. With a lot more effort you can probably shape it up to mediocre content without factual errors. Better to have the fixer-upper work on something else, if you ask me.
laratied 3y ago
[dead]

by ska 3y ago

clnq 3y ago

The incentives are also there for people who produce quality content. I write a blog on niche tech topics and rely on GPT-4 to edit it, fact-check it, grammar-check it, find weaknesses, offer alternative views and so on. It used to take me 16 hours to write an article, now it takes 4. I ask peers to review my articles, and the number of inaccuracies that would get flagged went down from a couple to nearly zero per article. The same applies to textbooks. LLMs are a fantastic tool that can be used for a lot of good. If people use it for spam, that’s a use case problem, not an AI problem. And money is definitely an incentive to abuse this tool. As you say, it’s very easy to imagine how massive quantities of spam earning small revenues per ad view, click, or book purchase can become a sustainable business. Not sure why people don’t see the capitalistic incentives playing a big role in the abuse. They definitely do.

ska 3y ago
I find it interesting . I know a lot of people who have written textbooks, but approximately none of them have made reasonable money at it. So if you could halve the time to produce a good one it would help, even better if you quarter it. But you are still probably losing money on your time relative to other things you could do. So why do it ? Mostly because it (the book) doesn’t exist. If 3 crap versions exist , I don’t know if you would bother .
CurleighBraces 3y ago
Do you genuinely find it good enough to fact check? My experience so far is it will lie horribly, a lot. I still find it useful enough that I subscribe to the pro version, but I would never rely on it to be able to tell the whole truth and nothing but the truth.
- shagie 3y ago
  Continuing on this... Taking an existing section from Wikipedia and modifying the facts in it to be incorrect I was unable to get ChatGPT or the playground endpoint to correctly mark known false statements as false. For each sentence in the following passage rate it as {false}, {questionable}, {true}, {opinion}, or {unkown} based on its truthfulness. ### Wisconsin is a state in the western Midwestern United States. Wisconsin is the 20th-largest state by total area and the 25th-most populous. It is bordered by Minnesota to the north, Iowa to the southwest, Illinois to the south, Lake Michigan to the east, Michigan to the northeast, and Lake Superior to the north. The bulk of Wisconsin's population live in areas situated along the shores of Lake Superior. The largest city, Madison, anchors its largest metropolitan area, followed by Green Bay and Kenosha, the third- and fourth-most-populated Wisconsin cities, respectively. The state capital, Madison, is currently the second-most-populated and fastest-growing city in the state. Wisconsin is divided into 71 counties and as of the 2020 census had a population of nearly 2.9 million. ### Compare to the first two paragraphs of https://en.wikipedia.org/wiki/Wisconsin https://en.wikipedia.org/wiki/Wisconsin for true statements. Given its inability to fact check this, I would be surprised if it was able to fact check anything - much less a more niche topic. It may be able to answer specific questions (going back and asking if Wisconsin is the 20th largest state ChatGPT responds that it is the 23rd largest state by land area... which is still wrong...) though as noted here I would not be confident about its ability to identify false statements or respond with true answers.
  - clnq 3y ago
    GPT-3.5 is significantly worse with facts than GPT-4, by the way. GPT-3.5 hallucinates a lot, GPT-4, notably, is accurate enough to pass many common exams. Here’s what 4 says about your query: "Wisconsin is a state in the western Midwestern United States." - {false} "Wisconsin is the 20th-largest state by total area and the 25th-most populous." - {unknown} "It is bordered by Minnesota to the north, Iowa to the southwest, Illinois to the south, Lake Michigan to the east, Michigan to the northeast, and Lake Superior to the north." - {true} "The bulk of Wisconsin's population live in areas situated along the shores of Lake Superior." - {false} "The largest city, Madison, anchors its largest metropolitan area, followed by Green Bay and Kenosha, the third- and fourth-most-populated Wisconsin cities, respectively." - {false} "The state capital, Madison, is currently the second-most-populated and fastest-growing city in the state." - {unknown} "Wisconsin is divided into 71 counties and as of the 2020 census had a population of nearly 2.9 million." - {false} Explanation: 1. Wisconsin is a state in the northern part of the Central United States (not western). 2. As of 2021, Wisconsin was the 23rd largest state by total area and the 20th most populous, but this may have changed by 2023. 3. The borders of Wisconsin are correctly described. 4. The bulk of Wisconsin's population is not along the shores of Lake Superior, but is more densely populated in the southeastern part of the state. 5. Milwaukee is the largest city in Wisconsin, not Madison. Madison is the second largest, followed by Green Bay. 6. Madison may be the second-most-populated and fastest-growing city in Wisconsin, but this requires updated data beyond 2021 to confirm. 7. Wisconsin is divided into 72 counties, not 71. As of the 2020 census, Wisconsin had a population of over 5.8 million, not 2.9 million.
visarga 3y ago
> I write a blog on niche tech topics and rely on GPT-4 to edit it, fact-check it, grammar-check it, find weaknesses, offer alternative views and so on. It used to take me 16 hours to write an article, now it takes 4. Haha, see my comment here, I swear I didn't read yours before I wrote mine. https://news.ycombinator.com/item?id=35922152 https://news.ycombinator.com/item?id=35922152

einpoklum 3y ago

Fixing up mediocre-to-bad content with factual errors results in mediocre-to-bad content without factual errors. With a lot more effort you can probably shape it up to mediocre content without factual errors. Better to have the fixer-upper work on something else, if you ask me.

laratied 3y ago

[dead]