4 ms·

Former reporter hear who has been in professional talks with a natural-language generation firm, dabbles in NLP, and has some insight into this process. A lot

by vonnik 12y ago

Former reporter hear who has been in professional talks with a natural-language generation firm, dabbles in NLP, and has some insight into this process. A lot of what machines churn out is based on templates that humans have created. Many of the things we read that are supposedly written by machines required major human intervention, and the application of strict constraints on the machine so that it wouldn't screw up. Number stories about baseball and the stock market are good for machines to write about. Most qualitative things aren't, for the moment. Even Google's results generating captions from images last November actually included a lot of variance. The algorithm was good at recognizing some things and not others. To the commenters on this thread who criticize news organizations, I would simply ask: Do you consume your news for free? Do you believe that synthesizing complex events using multiple sources of data happens without cost? Do you believe that news organizations pay reporters much to write stories? Please reflect on your role in the news ecosystem. Most readers today are free riders, and unwilling to subsidize quality journalism. That's one reason why it's dying.

TheSpiceIsLife 12y ago
Slightly tangential: while everyone's banging on a lot recently about how 'quality journalism is dying', has anyone actually run the numbers - has anyone done a quantitative analysis? Was it really the case that 20 or 30 years ago, and early, all / most journalism was high quality, or was it mostly trash back then too? And do we really have proportionately less quality journalism now that we did 'back then'? If we take in to account all the amazing podcasts we have access to now (eg. 99% Invisible, Australia's ABC Radio National podcasts, etc.) and the NYT, the New Yorker, Saudi Aramco News, etc etc, I'd hazard a guess there's actually a lot more quality journalism kicking about now than there ever was at any time in human history. (Edited to add: at least in this particular human's short history) I'm just stumbling around in the dark here though. Someone should do a Steven Pinker on this aphorism.
- vonnik 12y ago
  I respect your contrarian tendency, and have only anecdotal evidence to offer. I spent about 10 years abroad as a foreign correspondent and editor for some major publications. During that time, I saw the foreign bureaux of most major US news organizations eviscerated. Staff was chopped, layoffs every year, offices consolidated, many newspapers simply gave up on covering foreign news with people on the ground. That was a really disappointing time. There's always been a lot of trash in journalism. And there's still a lot of high-quality stuff. But as the news changes, there's a structural problem is measuring how good it is. Only newsmakers (the people behind the closed doors negotiating and making decisions) know what's really going on. Only they possess the test set to measure the reporting against. And unless there's an investigative reporter who has built relationships with them and convinces them to talk, the public will never know what actually happened, or know what public figures are hiding. The substitute for great investigative reporting isn't necessarily bad journalism. It's usually silence. How do you measure that? How do you know that no one wrote a story about something that you weren't conscious of? You don't. You're simply unaware. And that makes the parasitic special interests undermining the general welfare of this country and every other incredibly happy. There's a reason why investigative reporting is the first to fall: It's incredibly costly in terms of time, money, effort and even social relationships. News organizations and publishers put their reputations and networks on the line every time they blow the lid off a huge story. Reporters work for months cultivating sources' trust and learning where to look for and understand obscure documents. This kind of reporting doesn't make sense now that ad dollars have moved to Google. It doesn't have a place at Buzzfeed or Upworthy, because it's a huge risk and doesn't necessarily drive traffic, even if it impacts policy and elections.
jcoffland 12y ago
And what have you paid for 95% of the software you use? I'm guessing you're a free rider.
- dang 12y ago
  > I'm guessing you're a free rider. No personal attacks on Hacker News, please.
lucb1e 12y ago
> A lot of what machines churn out is based on templates that humans have created. Many of the things we read that are supposedly written by machines required major human intervention That's what I was thinking as I read through the samples. Computers do not yet have feelings or a proper emulation of them (that I know of, and it would sure be news) so ways of describing something must be from what the programmer put in. I can make a computer write stuff Shakespeare wrote, but it's still originally Shakespeare's (or mine, if I'm the programmer and thought of it). Bad quiz.
nodata 12y ago
Journalism is dying because small-ads disappeared. The masses never paid for quality journalism. Classified advertising revenue decline: http://www.journalism.org/media-indicators/newspaper-classified-ad-revenue/ http://www.journalism.org/media-indicators/newspaper-classif... With a better chart and commentary: http://www.businessinsider.com/chart-of-the-day-newspapers-classified-ads-revenue-2011-3?IR=T http://www.businessinsider.com/chart-of-the-day-newspapers-c... Expanded for all advertising revenue: https://gigaom.com/2013/04/11/two-charts-that-tell-you-everything-you-need-to-know-about-the-future-of-newspapers/ https://gigaom.com/2013/04/11/two-charts-that-tell-you-every...
hammerandtongs 12y ago
Just as a point of reference - http://www.wired.com/2015/03/disney-magicband/ http://www.wired.com/2015/03/disney-magicband/ This is journalism WE should pay for? Looks bought and paid for already...

by vonnik 12y ago

TheSpiceIsLife 12y ago

Slightly tangential: while everyone's banging on a lot recently about how 'quality journalism is dying', has anyone actually run the numbers - has anyone done a quantitative analysis? Was it really the case that 20 or 30 years ago, and early, all / most journalism was high quality, or was it mostly trash back then too? And do we really have proportionately less quality journalism now that we did 'back then'? If we take in to account all the amazing podcasts we have access to now (eg. 99% Invisible, Australia's ABC Radio National podcasts, etc.) and the NYT, the New Yorker, Saudi Aramco News, etc etc, I'd hazard a guess there's actually a lot more quality journalism kicking about now than there ever was at any time in human history. (Edited to add: at least in this particular human's short history) I'm just stumbling around in the dark here though. Someone should do a Steven Pinker on this aphorism.

vonnik 12y ago
I respect your contrarian tendency, and have only anecdotal evidence to offer. I spent about 10 years abroad as a foreign correspondent and editor for some major publications. During that time, I saw the foreign bureaux of most major US news organizations eviscerated. Staff was chopped, layoffs every year, offices consolidated, many newspapers simply gave up on covering foreign news with people on the ground. That was a really disappointing time. There's always been a lot of trash in journalism. And there's still a lot of high-quality stuff. But as the news changes, there's a structural problem is measuring how good it is. Only newsmakers (the people behind the closed doors negotiating and making decisions) know what's really going on. Only they possess the test set to measure the reporting against. And unless there's an investigative reporter who has built relationships with them and convinces them to talk, the public will never know what actually happened, or know what public figures are hiding. The substitute for great investigative reporting isn't necessarily bad journalism. It's usually silence. How do you measure that? How do you know that no one wrote a story about something that you weren't conscious of? You don't. You're simply unaware. And that makes the parasitic special interests undermining the general welfare of this country and every other incredibly happy. There's a reason why investigative reporting is the first to fall: It's incredibly costly in terms of time, money, effort and even social relationships. News organizations and publishers put their reputations and networks on the line every time they blow the lid off a huge story. Reporters work for months cultivating sources' trust and learning where to look for and understand obscure documents. This kind of reporting doesn't make sense now that ad dollars have moved to Google. It doesn't have a place at Buzzfeed or Upworthy, because it's a huge risk and doesn't necessarily drive traffic, even if it impacts policy and elections.

jcoffland 12y ago

And what have you paid for 95% of the software you use? I'm guessing you're a free rider.

dang 12y ago
> I'm guessing you're a free rider. No personal attacks on Hacker News, please.

lucb1e 12y ago

> A lot of what machines churn out is based on templates that humans have created. Many of the things we read that are supposedly written by machines required major human intervention That's what I was thinking as I read through the samples. Computers do not yet have feelings or a proper emulation of them (that I know of, and it would sure be news) so ways of describing something must be from what the programmer put in. I can make a computer write stuff Shakespeare wrote, but it's still originally Shakespeare's (or mine, if I'm the programmer and thought of it). Bad quiz.

nodata 12y ago

Journalism is dying because small-ads disappeared. The masses never paid for quality journalism. Classified advertising revenue decline: http://www.journalism.org/media-indicators/newspaper-classified-ad-revenue/ http://www.journalism.org/media-indicators/newspaper-classif... With a better chart and commentary: http://www.businessinsider.com/chart-of-the-day-newspapers-classified-ads-revenue-2011-3?IR=T http://www.businessinsider.com/chart-of-the-day-newspapers-c... Expanded for all advertising revenue: https://gigaom.com/2013/04/11/two-charts-that-tell-you-everything-you-need-to-know-about-the-future-of-newspapers/ https://gigaom.com/2013/04/11/two-charts-that-tell-you-every...

hammerandtongs 12y ago

Just as a point of reference - http://www.wired.com/2015/03/disney-magicband/ http://www.wired.com/2015/03/disney-magicband/ This is journalism WE should pay for? Looks bought and paid for already...