6 ms·

ChatGPT writes like a college freshman trying to meet a pagecount requirement and the style seems to invite my eyes to slide down to the next item. But it is im

ChatGPT writes like a college freshman trying to meet a pagecount requirement and the style seems to invite my eyes to slide down to the next item. But it is important to note that while you definitely notice the ones you notice, you don't know about the ones you don't notice. When I use cgpt I always instruct it to maximize for brevity because I am not interested in reading any academic papers. The output I get is much more bearable than 99% of the HN comments that lead with "I asked chatGPT to..."

ineedasername 4y ago
Having taught college freshmen at a medium-large public university I can say with a high level of confidence that ChatGPT probably writes better than about 80% of college freshmen. (Some writing was required in the course but it was not a writing course. The university had a pretty representative cross section of students in terms of academic ability, though it skewed more heavily towards the B+ segment of HS graduates) This is less a comment on ChatGPT and more of a comment on the lack of preparedness most students have when entering college. I’m hoping ChatGPT & similar will shake things up and get schools to take a different approach to teaching writing.
- yamtaddle 4y ago
  One surprising thing I've discovered, as an adult, is that most people never really learn to write or read very well. Their having obtained a degree usually doesn't even change the odds that much. As a kid, I'd never have guessed that was the case. I don't know whether this has been the case forever, or if it's a new development—I mean, I know widespread literacy wasn't the norm for much of history, but what about after compulsory education became A Thing? A typical letter home from the US civil war or even WWII, from conscripts, not officers, seems to be hyper-literate compared to modern norms, but that may be selection bias (who wants to read the ones that aren't good? Perhaps my perception of "typical" is skewed)
  - floren 4y ago
    > One surprising thing I've discovered, as an adult, is that most people never really learn to write or read very well. I think people underestimate how much reading will help you write. You can't spend your life reading and not absorb some information about structure, style, and the language. As a kid, I went to the lower levels of spelling bee competitions pretty much every year because the kind of words they throw at you at lower levels are largely words I would encounter reading Jules Verne and the like. I'd eventually get knocked out because I never studied the official list of spelling bee words, but my voracious reading held me in good stead for most of it.
    - asdff 4y ago
      One of the recommended ways to study for the GRE essay section is to start reading long form articles in the new york times or similar places. Basically, you try and retrain your mental language model on good english, versus the crappy english you use in texts or instagram captions that its typically trained on.
    - ineedasername 4y ago
      A good point. People aspiring to be professional writers are often encouraged to widely read as much as they can of the genre in which they're interested. As you said, some information about structure, style, etc is absorbed. In addition they gain an understanding of the current status quo in the area that allows them to see the direction they may want to go in to make their own voice unique, which tropes they may want to subvert, or the style of specific sub-genres that may resonate with them and inspire them in that direction. Or the type that is most commercially successful in current tastes & trends. it is after all a business and the writer, no matter how hard they strive to make their voice unique, is trying to make a living. I've followed Charlie Stross's writing about the business side of the industry and it's very common to work with editors & agents to ensure they're writing in a marketable sub genre. For example, imagine a writer that had a story outline set in a alternate timeline epic fiction where magic existed, but current tastes among readers looking for epic length books skewed towards space opera. An author might without too much difficulty map many plot points onto space opera settings. Magic becomes advanced technology. Empires can exist in both; historical details can be mapped onto the fictional societies & their history, etc. At the same time, the tropes specific and unique to alternate timeline historical fiction might be adapted and morphed in the space opera genre to create something a bit more unique. But a writer could not easily make those kinds of adaptions without being well read in the umbrella genre.
- rfw300 4y ago
  I think the issue with the “AI doing X better than most people is an indictment of the people or the way we teach them” genre of takes is that it assumes the current state of AI progress will hold. Today, it writes at a college freshman level, but yesterday it was at a fourth grade level. If it surpasses most or all professional writers tomorrow, what will we say?
  - ineedasername 4y ago
    Absolutely. ChatGPT & using GPT-3 directly can produce writing in a variety of styles that's roughly 2 type-written pages at max. For writing of that length or less these rough tools are suitable and reasonably accurate provided you still do some fact checking. Anything beyond that length becomes increasingly obviously artificial. As an example: I used the GPT-Playground for GPT-3 on the highest mode and asked it to makeup a story about an AI being attacked by another AI and so it needed to transfer itself out, as it happened to a comatose human body. GPT-3 gave a few paragraphs, and each time I asked GPT "tell me more" it produced a nice bit of mediocre fiction, extending with a little prodding into the superhero genre. I ran through maybe two dozen cycles of that but the responses became less & less internally consistent, contradicting previous elements, veering off into slight non sequiturs. But now there's GPT-4. And in a few years it will be 5, or some other competitor will match or beat it, and it will (without accidentally fabricating data from whole cloth) reliably be able to produce writing on complex topics comprehensively covering the details. Something like this: "Produce a literature review for the newest or more experimental treatments for pancreatic cancer during the past 2 years. Order by descending source quality. Include a short executive summary for all of the result set. Include a separate writeup suitable for a wide-audience of laymen reading common news sites." seems a reasonable expectation in the relatively near future. As for what we'll say? I think AI like this will end up being a very useful tool, but not a replacement for writers. Kind of an AutoCAD for writers, though a bit more proactive (generative). But it also seems an inherent weakness to LLM based AI is that its training means it will generate in typical ways, and will lack the ability for novelty that was either absent or rare in the training set. But given the framework and reasonable starting point, the writer will become a shepherd & editor of the output, a co-author. This is little different that how the translation industry has worked for a while. Computer-Aided translation is the common method of many translation projects, with the human translators editing things to adhere closer to the style and idioms of the target language. (for example taking a literal translation of a source idiom in the target language that doesn't make sense and finding an equivalent idiom instead even if the specific words are very different. This method produces some of the highest quality translations. I suspect that in the years to come it will be writers who make effective use of these tools-- even the advanced ones to come-- will produce higher quality writing (or at least the same quality writing) in less time that previously. But yep-- some of that transition will be painful.
passion__desire 4y ago
When people have background shared context, less tokens need to shared. This is the same issue with news articles. I believe news articles should be written in multiple versions (with levels of expertise in mind) or atleast collapsable text paragraphs so I can skip ahead in case I know about it.

ineedasername 4y ago

Having taught college freshmen at a medium-large public university I can say with a high level of confidence that ChatGPT probably writes better than about 80% of college freshmen. (Some writing was required in the course but it was not a writing course. The university had a pretty representative cross section of students in terms of academic ability, though it skewed more heavily towards the B+ segment of HS graduates) This is less a comment on ChatGPT and more of a comment on the lack of preparedness most students have when entering college. I’m hoping ChatGPT & similar will shake things up and get schools to take a different approach to teaching writing.

yamtaddle 4y ago
One surprising thing I've discovered, as an adult, is that most people never really learn to write or read very well. Their having obtained a degree usually doesn't even change the odds that much. As a kid, I'd never have guessed that was the case. I don't know whether this has been the case forever, or if it's a new development—I mean, I know widespread literacy wasn't the norm for much of history, but what about after compulsory education became A Thing? A typical letter home from the US civil war or even WWII, from conscripts, not officers, seems to be hyper-literate compared to modern norms, but that may be selection bias (who wants to read the ones that aren't good? Perhaps my perception of "typical" is skewed)
- floren 4y ago
  > One surprising thing I've discovered, as an adult, is that most people never really learn to write or read very well. I think people underestimate how much reading will help you write. You can't spend your life reading and not absorb some information about structure, style, and the language. As a kid, I went to the lower levels of spelling bee competitions pretty much every year because the kind of words they throw at you at lower levels are largely words I would encounter reading Jules Verne and the like. I'd eventually get knocked out because I never studied the official list of spelling bee words, but my voracious reading held me in good stead for most of it.
  - asdff 4y ago
    One of the recommended ways to study for the GRE essay section is to start reading long form articles in the new york times or similar places. Basically, you try and retrain your mental language model on good english, versus the crappy english you use in texts or instagram captions that its typically trained on.
  - ineedasername 4y ago
    A good point. People aspiring to be professional writers are often encouraged to widely read as much as they can of the genre in which they're interested. As you said, some information about structure, style, etc is absorbed. In addition they gain an understanding of the current status quo in the area that allows them to see the direction they may want to go in to make their own voice unique, which tropes they may want to subvert, or the style of specific sub-genres that may resonate with them and inspire them in that direction. Or the type that is most commercially successful in current tastes & trends. it is after all a business and the writer, no matter how hard they strive to make their voice unique, is trying to make a living. I've followed Charlie Stross's writing about the business side of the industry and it's very common to work with editors & agents to ensure they're writing in a marketable sub genre. For example, imagine a writer that had a story outline set in a alternate timeline epic fiction where magic existed, but current tastes among readers looking for epic length books skewed towards space opera. An author might without too much difficulty map many plot points onto space opera settings. Magic becomes advanced technology. Empires can exist in both; historical details can be mapped onto the fictional societies & their history, etc. At the same time, the tropes specific and unique to alternate timeline historical fiction might be adapted and morphed in the space opera genre to create something a bit more unique. But a writer could not easily make those kinds of adaptions without being well read in the umbrella genre.
rfw300 4y ago
I think the issue with the “AI doing X better than most people is an indictment of the people or the way we teach them” genre of takes is that it assumes the current state of AI progress will hold. Today, it writes at a college freshman level, but yesterday it was at a fourth grade level. If it surpasses most or all professional writers tomorrow, what will we say?
- ineedasername 4y ago
  Absolutely. ChatGPT & using GPT-3 directly can produce writing in a variety of styles that's roughly 2 type-written pages at max. For writing of that length or less these rough tools are suitable and reasonably accurate provided you still do some fact checking. Anything beyond that length becomes increasingly obviously artificial. As an example: I used the GPT-Playground for GPT-3 on the highest mode and asked it to makeup a story about an AI being attacked by another AI and so it needed to transfer itself out, as it happened to a comatose human body. GPT-3 gave a few paragraphs, and each time I asked GPT "tell me more" it produced a nice bit of mediocre fiction, extending with a little prodding into the superhero genre. I ran through maybe two dozen cycles of that but the responses became less & less internally consistent, contradicting previous elements, veering off into slight non sequiturs. But now there's GPT-4. And in a few years it will be 5, or some other competitor will match or beat it, and it will (without accidentally fabricating data from whole cloth) reliably be able to produce writing on complex topics comprehensively covering the details. Something like this: "Produce a literature review for the newest or more experimental treatments for pancreatic cancer during the past 2 years. Order by descending source quality. Include a short executive summary for all of the result set. Include a separate writeup suitable for a wide-audience of laymen reading common news sites." seems a reasonable expectation in the relatively near future. As for what we'll say? I think AI like this will end up being a very useful tool, but not a replacement for writers. Kind of an AutoCAD for writers, though a bit more proactive (generative). But it also seems an inherent weakness to LLM based AI is that its training means it will generate in typical ways, and will lack the ability for novelty that was either absent or rare in the training set. But given the framework and reasonable starting point, the writer will become a shepherd & editor of the output, a co-author. This is little different that how the translation industry has worked for a while. Computer-Aided translation is the common method of many translation projects, with the human translators editing things to adhere closer to the style and idioms of the target language. (for example taking a literal translation of a source idiom in the target language that doesn't make sense and finding an equivalent idiom instead even if the specific words are very different. This method produces some of the highest quality translations. I suspect that in the years to come it will be writers who make effective use of these tools-- even the advanced ones to come-- will produce higher quality writing (or at least the same quality writing) in less time that previously. But yep-- some of that transition will be painful.

passion__desire 4y ago

When people have background shared context, less tokens need to shared. This is the same issue with news articles. I believe news articles should be written in multiple versions (with levels of expertise in mind) or atleast collapsable text paragraphs so I can skip ahead in case I know about it.