17 ms·
> So I threw in the entire text into Hemingway, an app that identifies poor or confusing writing. I found this interesting, because it seemed unbelievable to m
by aerovistae 6y ago
> So I threw in the entire text into Hemingway, an app that identifies poor or confusing writing.
I found this interesting, because it seemed unbelievable to me that an app could be able to tell good writing from bad writing in a reliable or useful way.
So I took excerpts from Shirley Jackson's The Lottery and pasted them in. (For anyone unfamiliar with it, The Lottery is considered a masterpiece of a short story, and is widely taught in American high schools.) The excerpts I pasted were almost universally painted red: bad writing, "very hard to read."
Then I took excerpts of Isaac Asimov's The Last Question, another celebrated English-language short story in a very different writing style. Red and yellow all over.
Knowing full well what it would think of James Joyce, I skipped over him to try Roald Dahl's The Great Automatic Grammatizator -- red and yellow all over.
Lastly, for the sake of variety, I decided to try something simple and put in large swaths of one of the original Winnie the Pooh books. Finally, something it was mostly okay with! But only the dialogue. Anytime the author dared to write any description, red all over.
Seriously, don't use this tool.
Edited perspective from comment replies to mine: the app is largely okay with Hemingway, which makes sense as its main intent is to make your writing like Hemingway's. Also, if it is largely intended to be for business writing, then it makes sense that it would react poorly to literary writing. Both good points!
- 6gvONxR4sf7o 6y agoIt’s a great example of how people will use an easy quantification just because it exists. There’s a popular mindset that seems to think you can’t argue against numbers and math because they’re cold and scientific. Who cares if its a stupid number, it’s mathematically/scientifically correct!
- keiferski 6y agoNew tools, same old tricks. We've just upgraded from turtle shells to web apps.
- jimmaswell 6y agoA measure that becomes a target often stops being a good measure.
- jjoonathan 6y agoYes, but sometimes it was garbage in the first place, too.
- ketzo 6y agoHuh. This is actually quite insightful to me. I wonder if this applies more broadly? What from your own life makes you say that?
- InitialLastName 6y agoNot GP, but that's a formation of Goodhart's law [0], which comes up in a lot of situations. Notable potential Goodhart's law situations: - Testing in education - GDP (or a number of other economic indicators) as a measure of economic performance/target for economic performance - Healthcare outcomes along a number of axes [0] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- banana_giraffe 6y agoIt's a variant of Goodhart's law. Can't speak for the person you're replying to, but absent good management, I've seen review goals turn into nonsense. "You had a goal to fix 5 bugs this quarter, and only fixed 4", "but one was a massive bug that would have cost us $x!", "but you didn't fix 5 bugs, so no bonus" You see it happen on different levels a surprising number of times.
- indigochill 6y agoAnecdotally, when I worked in customer support, we were explicitly told that we had to hit a certain number of replies to customers in our ticket system. So I played by the rules: I very clearly grabbed all the easy-to-reply tickets, sent the easy reply, and dropped them if they got hard. Appropriately, a manager spoke to me telling me this was unacceptable (though he did concede the goal was stupid), and gratifyingly the official goals soon changed to reflect an appropriate balance of consistency with a customer as well as some degree of overall throughput. The drawback there is that accurately reflecting what they want in hard metrics might not really be possible because it does come down to a judgment call where the exact balance between support quality and throughput lies (and it's often a shifting balance as well, depending how much support load there is). Which is probably why managers fall back on stupid but easy-to-measure metrics.
- africanboy 6y agothat's not how it works though... you can't argue against numbers, you can absolutely argue against arbitrary numbers (like in this example the premise is that the app produces reliable results, which it does not and anyway it should be proved with good, hard to dispute, numbers)
- Barrin92 6y agoOP's point is that all numbers are arbitrary (to varying degrees). The act of quantifying always imposes some sort of framework on what you're trying to look at, and we live in an age of pretty vulgar scientism where this is associated with 'objectivity'. From 'personality scores' for applicants, to social credit scores, and in this case the quality of writing in a newspaper.
- 6gvONxR4sf7o 6y agoRight. The number 3 is inarguable. That 4+2=6 is inarguable. The applicability of a give statistic to a given situation or argument is a much more difficult proposition, but people too frequently treat a statistic as relevant just because it's easy to compute. After all mercury is in retrograde, and that's a scientific fact.
- africanboy 6y agoand they are all non arbitrary (to varying degrees), but arguably some are objectively more arbitrary than others.
- anamexis 6y agoI have no intention of using Hemingway or anything like it, but I think it is obviously targeted at business communication, not literature, so it's unsurprising that it would find issues with literature excerpts.
- aaron-santos 6y agoYou should see what it does to The Old Man And The Sea! 8 of 263 sentences are hard to read. 15 of 263 sentences are very hard to read.
- TravisLS 6y agoI like the Hemingway app, but it is very much designed to make you write like Hemingway (ie short, clear sentences _only_). Obviously most great writing is not in this style. The Hemingway app is good for work emails and marketing copy, or if you happen to like that style. It is hardly the universal arbiter of "poor or confusing writing".
- aerovistae 6y agoOh wow it's almost completely okay with Hemingway. I pasted some of his work in and it had almost no complaint at all. Good point.
- effingwewt 6y agoI was coming to say the same, and the descriptors being removed actually just points mote to its usefulness for writing 'in a Hemingway style'. However, having said that I don't think the author of this article used it well, as for many reasons most any journalistic article is going to be very descriptive and use things like puns to get/maintain a reader's attention (something I personally usually hate, but not always).
- abvdasker 6y agoI think it would be fun to create a version of this which pushes writing towards other recognizable styles like Raymond Chandler or David Foster Wallace. Maybe you pick a style from a dropdown and the analysis changes. You could possibly take it even further and automate some transformations of the input to stylize a piece of writing.
- chrisdhal 6y agoTo satisfy a DFW style it'd need to understand 2 page long sentences with multiple footnotes per sentence. Haha. It would be a fun one to do though! (I say that loving DFW)
- varjag 6y agoIt's Hot Dog/Not Hot Dog but for creative writing.
- ghaff 6y agoI sometimes write for one of the online tech pubs. The tool you publish through has a reading difficulty score which I always fail. (I mostly ignore it; at most I might tweak a couple of things.) Also all sorts of SEO-related stuff.
- cowboysauce 6y ago> Knowing full well what it would think of James Joyce, I skipped over him I like to imagine that Finnegans Wake would result in a crash.
- forrestthewoods 6y ago> Seriously, don't use this tool. Horrible take. I use the tool for all of my blog posts. It’s a stellar tool with a couple of killer features. I 100% promise if you the tool it will improve the quality of your writing. That said, it does not identify “good writing” or “bad writing”. It is not sacrosanct. And it definitely can’t write for you. It’s perfectly valid to leave in yellow or red sentences. But the tool helps make it a conscious decision when to do so.
- Veen 6y agoIt’ll improve your writing if you’re a lazy, inattentive, careless writer. If you have any writing ability and pay attention to what you are doing, it’s worthless.
- forrestthewoods 6y agoI bet you don’t use spell checkers or intellisense either Feel free to read my blog. I used Hemingwayapp on almost every post. I’ll let you decide if I’m a lazy, inattentive, careless writer. https://www.forrestthewoods.com/blog/ https://www.forrestthewoods.com/blog/
- 8fGTBjZxBcHq 6y agoSo I don't know anything about this app really or its evaluations but I think applying it to acclaimed fiction is not a criticism of applying it to a newspaper article per se. News articles are meant to convey concrete information relatively concisely. So metrics like sentence length and complexity and adverb count are at least in theory valid things for it to measure and aim to reduce. Now whether it effectively measures those things I have no idea or opinion about. I didn't know it existed until ten minutes ago but if I had I definitely would have predicted that most highly respected fiction would break its rules all over the place.
- deleted 6y ago[deleted]
- matthewowen 6y agoThis isn't a news article: it's a column, written by someone who's explicit remit is columns about the media industry. It absolutely conforms to different rules than a news article on page A1. I mean, I think it's bad writing too! But it'd be bad writing if it were in something like the New Yorker, too.
- bko 6y agoYes, the tool isn't great. But it provides a sanity check to confusing run on sentences or use of adverbs and passive voice. I also pasted the first page of The Lottery, and it scored it a Grade of 11, Readability: OK, compared to a 14 on the Times article. Do you find the times article mentioned easy to read or well written?
- ghaff 6y agoThat Times article is not great. Not sure if I'd have made it through the first paragraph. That said, these types of tools often seem to be about dumbing things down to increase "engagement."
- sleepydog 6y agoFor kicks, I threw a few paragraphs from The old man and the sea to see if the editor displayed any deference to its namesake. Red all over, "very hard to read", "use of the passive voice".
- aerovistae 6y agoI didn't try that one-- I put in Hills Like White Elephants and it was mostly okay.
- Nullabillity 6y agoYup, any kind of automatic quality assessment tool is made with the assumption that quality is even something that can be measured objectively. Zoe Bee recently made a pretty good video about why this falls completely flat for grammar (rather than just appealing to "X is good, Y disagrees, Y must be wrong"): https://www.youtube.com/watch?v=Q5rB9jDbTPU https://www.youtube.com/watch?v=Q5rB9jDbTPU
- TheOtherHobbes 6y agoHemingway is designed for genre fiction - in fact a certain terse style of genre fiction - not for journalism. Although I somewhat agree with the premise, especially with science and tech journalism. Most is fine, but every so often there's an article which looks like it was written by someone for whom English is a fifth language. Blog posts can be hit and miss too, but that's not such a surprise.
- Spivak 6y agoI think you’re misunderstanding this tool. You’re shoving a Monet into a tool for graphic designers and complaining that it flags it as blurry. Writing for news should be boring, dry, precise, clear and near painfully explicit with as little subtext as can possibly be achieved. Hemingway’s writing style just happens to be a good example of it. I think you and people in this thread are confusing news with storytelling/newstainment like John Oliver, lots of Gimlet, Vice, etc. who take the news and up the production value by infusing their personalities and commentary. Despite the crass name this is a valuable service. Good storytellers are the bridge between the dry academic work of historians, journalists, scientists, philosophers, etc. and people outside their respective fields. tl;dr news isn’t art and can’t be judged by the same measures.
- ghaff 6y agoSo you don't like the writing in The Economist? I do think it aims to be precise and clear but I certainly wouldn't say it aims to be boring.
- akudha 6y agoCan anyone shed light on how it works? There is another one that comes to mind, PTE exam. The PTE exam is fully computer graded (as far as I know), including written essay. I wonder how they do it. The algorithm has to account for topic relevance, grammatical errors, style, clarity etc etc. Topic relevance alone can be quite hard, isn't it? If the question is "What is your opinion on making voting mandatory? Support with examples", I can talk about voting and elections without actually giving an opinion. How does the algo detect it? PTE, Hemingway app etc - really intriguing to reduce human language to logic and calculation.
- iudqnolq 6y agoYou might enjoy this researcher's automated high scoring essay generator > Our hypothesis was simple. If the AES machine consistently gave high scores to machine generated gibberish, we could surmise that 1) the construct being measured by the machines is not an essential component of human communication; and 2) students could be taught similar strategies to achieve high scores on computer scored writing tests by sprinkling their prose with long meaningless sentences composed of pretentious and irrelevant words. > Our greatest surprise was how easy it was to fool all of the machines. We succeeded on our first try, demonstrating that rather than being elegant and complex manifestations of state-of-the-art artificial intelligence, these engines could best be characterized as crude stupid machines. EDIT (an example): > We have now used the Babel Generator over twenty times to generate essays for the site, which, when submitted, receive top scores with comments such as articulates a clear and insightful position on the issue in accordance with the assigned task and sustains a well-focused, well-organized analysis, connecting ideas logically” for essays that read like this following opening paragraph: > Careers with corroboration has not, and in all likelihood never will be compassionate, gratuitous, and disciplinary. Mankind will always proclaim noesis; many for a trope but a few on executioner. A quantity of vocation lies in the study of reality as well as the area of semantics. Why is imaginativeness so pulverous to happenstance? The reply to this query is that knowledge is vehemently and boisterously contemporary. https://lesperelman.com/writing-assessment-robo-grading/babel-generator/ https://lesperelman.com/writing-assessment-robo-grading/babe...
- teh_klev 6y agoI probably agree about the tool, each writer adds their own flourishes and richness to their prose to capture vividly in your mind a scene or emotion. Hemingway probably doesn't understand this. But the reason he arrives at using it is because he rightly calls out a bunch of prose that is terrible writing and is trying to quantify this in some way. And whilst I have no problems reading long form articles, many these days do tend to suffer from confusing and downright tedious language.
- andi999 6y agoI am a bit surprised about: " The Lottery is considered a masterpiece of a short story, and is widely taught in American high schools." Really? I have difficulties understanding this. Is it because it is interesting for the kids through its shock value and manageable length? But this then looks to me like declaring ketchup to be vegetables so enough veggies are eaten at the canteen.
- aerovistae 6y agoWell, I suppose you could read some critical analysis of it if you're interested in why it's considered great; I'm no expert. For what it's worth I agree with its status in the canon: it's a powerful and carefully articulated story in a very compact form with a unique, exciting, and thought-provoking premise. The perfect short story.
- andi999 6y agoYes it is good entertainment, but wouldnt then the kids read those anyways?
- beanders 6y agoThat’s not a given, unfortunately. I believe that the intended long term strategy is to give kids a glimpse of something that is outwardly interesting, in order to increase their internal motivation to seek out other good reading material. Not necessarily to get it “right” the first time.
- ghaff 6y agoI don't claim to know the current thinking but, to me, it makes sense that the focus should be on "classic" literature (for which The Lottery qualifies) that's also approachable. I'd argue that a lot of Hemingway and Fitzgerald do as well--though I don't fully understand the popularity of The Old Man and the Sea. I liked reading in high school and most of the 19th and 18th century works I read bored me to tears and I probably ended up at least partly just reading the Cliff Notes. Today, I expect I would go to Wikipedia.
- coliveira 6y agoEnglish is not my first language, so it astonishes me that some people really think that short sentences make good writing. Anyone can write bad prose using short sentences. Moreover, by writing in this way the language loses any kind of spice, so it's not a surprise that journalism and technical texts are so unpalatable. Another consequence is that even educated people never stop sounding like they're talking to a ten year old.
- coliveira 6y agoBut in a sense I realize why this is a problem in English: German an some latin languages have grammatical structure to make it easier to create and understand long phrases. English lacks much of that structure. So, long phrases are difficult to parse. Compare for example Proust novels written in English and in his French original. In English it is almost gibberish, even in good translations.
- ghaff 6y agoThat's probably true. I would have had to work to parse the first sentence in that article. The other thing going on is that, for many situations, the use of punctuation within long sentences to help the reader has sort of fallen out of fashion relative to breaking long sentences up. It's probably also more acceptable to use sentence fragments in certain types of prose.
- com2kid 6y ago> The other thing going on is that, for many situations, the use of punctuation within long sentences to help the reader has sort of fallen out of fashion relative to breaking long sentences up. Semicolons, colons, and dashes, are rarely used now days. Heck I am impressed when I see a dash properly used, I myself cannot lay claim to such knowledge. I am almost 100% certain that their usage was not taught to me in either school or college.
- ghaff 6y agoI use em dashes a fair bit (and have to remember the spacing style of the publication I'm writing for). But I'm much more inclined to break a sentence in two rather than use a semicolon unless there are two very connected thoughts.
- sanderjd 6y agoBut I've always wondered: why should business writing be like Hemingway's writing? He wasn't writing for that audience. Seems like it would be better to find examples of good writing for a business audience and use those. Same thing for journalistic writing: that's not what Hemingway was doing. Maybe take all the Pulitzer winning journalism from a few decades and use that or something. But it just seems weird to pick one arbitrary literary writer for this kind of exercise.
- prepend 6y agoIt’s a good question and one I don’t know how to answer absolutely. The reason why I think it most generally applies to “business writing” is because this area is most likely to value efficiency over others. I don’t think it’s meant to say business folk should write like Hemingway, but that Hemingway is the most popular writer in this style.
- kevan 6y ago>>this area is most likely to value efficiency Agreed. The general style values simple sentences which helps reading speed and comprehension, especially for audiences that don't share your native language. And it reduces fluff. Adverbs, for example, generally don't help make a point in business writing - data does. See the example in that last sentence? I snuck a premise instead of stating it as a hypothesis or providing data to support it. At best that causes confusion for the reader and at worst it can mislead people that aren't carefully watching out for pitfalls like that.
- alisonkisk 6y agoWhat was the sneaky premise? A premise is a hypothesis. We don't explicitly tag all our premises; it's up to the reader to decide if they agree with the premises.
- kevan 6y agoFair, it wasn't a great example. >We don't explicitly tag all our premises True but we should aim to tag more of them in business writing. It's helpful to clearly state them so 1. You as the writer are crystal clear on the assumptions you're making, and 2. if there's any disagreement on premises you can resolve it before you spend any time discussing which course of action to take.
- mumblemumble 6y agoThat Shirley Jackson example is particularly galling because her prose is considered by many people to be about as close to perfect as a human can get. Better than Hemingway's. IIRC, her editor said in an interview that she was their easiest client to work with, because her work didn't actually need editing.
- jschwartzi 6y agoThe reading level is probably the best gauge for whether an average person will understand what you're saying. If it's reading level 14, that means post high school. Most people in the US graduate from high school with an 8th or 9th grade reading level. So a piece with a 14th-grade reading level will be completely beyond them.
- minimaxir 6y agoIt is somewhat ironic that a blog about machine learning cherry-picks results from a black-box model to prove a point.
- crochi 6y agoI believe red and yellow just mean the 'reading level' expected. When writing for a very broad audience, or writing things that need to be short and easy to read like e-mails, you want to aim for 'lower' reading levels. I pasted some excerpts from a paper I was reading and it gave the 'Post-Graduate' reading level. Does not mean it's bad writing, just hard to read (it's probably inevitable at that level). It's probably a great tool for daily work, some people at my workplace used Grammarly, but it's paid, so this could be a good alternative.
- kingofpandora 6y agoAny tool that uses passive voice as a heuristic for good versus bad writing is immediately suspect. And any writer who uses such a tool for validation is unlikely to be a good judge of writing to begin with.
- alisonkisk 6y agoIs what you meant that it is suspect to have passive voice being used as a heuristic?
- inglor_cz 6y agoWriter here, reasonably competent, because people pay me for writing (in Czech; forgive me my English errors). Passive voice in narration is ... strange most of the time. It kicks the reader out of the situation, like Out-of-body experience, only it is an Out-of-book experience. "John was killed by the assassins, his body was later discovered by a cleaning lady, carried out by morticians and dissected by pathologists." It is pretty hard to feel any emotions around this almost technically cold description. A sentence like this could have come from a police dossier. I am not a friend of software validation tools in this regard, though. Capturing the essence of beauty in words isn't a solved problem.
- tomcam 6y agoYour written English is superb. It's indistinguishable from that of a literate native speaker.
- hertzrat 6y agoWell said. It’s like people think they have to get the machine to validate them instead of the other way around. I was coaching somebody’s writing recently and trying to tell them not to modify their style based on Hemingway feedback, but they just had so much faith in it. Why would so many people use it if it wasn’t reliable, right?
- jasonkester 6y agoIndeed. When it first showed up here, I tried feeding some Hemingway to Hemingway. It hated it.
- SkyBelow 6y agoSounds like a set of training wheels. Helpful for those begging to learn, but extremely limiting if you don't learn to take them off once you have the basics. I can definitely see the benefit of such an application, but the risks are also quickly apparent. Looking at it from one level higher, I think I would be in favor of such an application because those who don't ever take the training wheels off will, on average, gain more than had they never had it while those who would be held back by it are likely to outgrow it on their own pace. The only real threat is if it becomes a tool of authority, like teachers that depend upon it and mark down writing that the app doesn't like.
- lliamander 6y agoSpecifically, Hemingway was a journalist. His style is one I would expect to be a reasonable standard for newspapers in particular.
- Kluny 6y agoDid you try throwing any Hemingway in there? :) I agree, Hemingway is a useful tool for editing, but it's not any kind of authority on what "good" or "bad" writing is. It's great for reducing your word count during final edit, but that's about it.
- deleted 6y ago[deleted]
- slaymaker1907 6y agoI watched https://www.youtube.com/watch?v=Q5rB9jDbTPU https://www.youtube.com/watch?v=Q5rB9jDbTPU yesterday which goes into details about problems with a similar service, Grammarly. What Bee said really resonated with me. I used to be a stickler for grammar rules and "proper" English, but as I've progressed in life I've come to realize that those rules are all secondary. All that really matters is that you are understood in the way you want to be understood. Grammar rules help educate people about common idioms and structures so that they can understand others and be understood themselves. Additionally, by following a certain set of grammatical conventions, you can change the way you are perceived. By ignoring the rules, one might be seen as erudite and sophisiticated. OTOH u dont always want to seem stuckup :)