4 ms·

I'm a former journalist and spent the first 10 years of my career at various magazines and newspapers. About eight years ago, I landed my first full-time softw

by jawns 8y ago

I'm a former journalist and spent the first 10 years of my career at various magazines and newspapers. About eight years ago, I landed my first full-time software development job in part because of a walkthrough I gave during my in-person interview process of a tool I'd developed while working for a metro daily newspaper. It wrote lottery results stories with the click of a button. Writing those results stories was a daily chore. We would look up the state's daily lottery results and also report Powerball and Mega Millions results. For those larger lotteries, we'd also adjust our headlines depending on whether there was a big winner in our state. But aside from that, it was just tedious and formulaic, so I wrote a script that would hit the various lottery sites, scrape the data, and generate a two- or three-paragraph story that was ready to post. It took the task down from five minutes to five seconds. (I later worked on a similar tool for generating short weather reports and alerts.) Since then, I've worked on other NLG stuff, and honestly, it's pretty hard, and the topic really has to be data-driven to begin with; we're probably decades away from really insightful NLG that offers "why" explanations, as opposed to "what" or "how" descriptions. The other tough challenge about computer-generated journalism is that getting the data is often the hardest part of the process. Oh, sure, businesses are going to release their quarterly earnings reports, and sports teams are going to release their game data. You might even use computer-assisted reporting to generate FOIA requests for data that governments are required by law to release upon request. But you're not going to write a story that leads to Nixon's resignation or Enron's collapse simply by asking for the data.

msla 8y ago
> we're probably decades away from really insightful NLG that offers "why" explanations, as opposed to "what" or "how" descriptions. For any specialized field, doing this would require something more than the kinds of schooling journalists get. It would also step on a lot of political toes: For example, the current national debt is $X. Why is it that big? Is that number a problem? Say anything concrete regarding either of those questions and you get screeching. Just endless, wordless screeching, like the Pod People in Invasion of the Body Snatchers, only directed into your Letters to the Editor column.
- neffy 8y ago
  To be fair to the journalists, those are actually open research questions.
  - msla 8y ago
    > To be fair to the journalists, those are actually open research questions. There are strong consensuses about some aspects of those problems, however, especially regarding the benefits of using government debt to finance stimulus programs as part of a counter-cyclical action against a recession. Saying it's all unknown is about as intellectually honest as saying that we don't know whether the world is flat or round because we're unsure of its topology at the millimeter scale.
    - bgorman 8y ago
      Economics, like all social sciences is nowhere near as sound as the hard sciences. You simply cannot have control groups for many experiments so you are left with case studies, which are often unconvincing. For example, does a true free market really exist anywhere in the world? How about a 100% command economy? How can you honesty determine if the federal reserve helps or hurts the economy?
      - msla 8y ago
        > Economics, like all social sciences is nowhere near as sound as the hard sciences. You simply cannot have control groups for many experiments so you are left with case studies, which are often unconvincing. You can say the same thing about paleontology, but do you seriously doubt the past existence of non-avian dinosaurs? > For example, does a true free market really exist anywhere in the world? How about a 100% command economy? You don't need perfect examples of things to observe what happens when economies closely approach those extremes. > How can you honesty determine if the federal reserve helps or hurts the economy? By observing how horrible the business cycle was prior to its existence.
        8y ago

by jawns 8y ago

msla 8y ago

> we're probably decades away from really insightful NLG that offers "why" explanations, as opposed to "what" or "how" descriptions. For any specialized field, doing this would require something more than the kinds of schooling journalists get. It would also step on a lot of political toes: For example, the current national debt is $X. Why is it that big? Is that number a problem? Say anything concrete regarding either of those questions and you get screeching. Just endless, wordless screeching, like the Pod People in Invasion of the Body Snatchers, only directed into your Letters to the Editor column.

neffy 8y ago
To be fair to the journalists, those are actually open research questions.
- msla 8y ago
  > To be fair to the journalists, those are actually open research questions. There are strong consensuses about some aspects of those problems, however, especially regarding the benefits of using government debt to finance stimulus programs as part of a counter-cyclical action against a recession. Saying it's all unknown is about as intellectually honest as saying that we don't know whether the world is flat or round because we're unsure of its topology at the millimeter scale.
  - bgorman 8y ago
    Economics, like all social sciences is nowhere near as sound as the hard sciences. You simply cannot have control groups for many experiments so you are left with case studies, which are often unconvincing. For example, does a true free market really exist anywhere in the world? How about a 100% command economy? How can you honesty determine if the federal reserve helps or hurts the economy?
    - msla 8y ago
      > Economics, like all social sciences is nowhere near as sound as the hard sciences. You simply cannot have control groups for many experiments so you are left with case studies, which are often unconvincing. You can say the same thing about paleontology, but do you seriously doubt the past existence of non-avian dinosaurs? > For example, does a true free market really exist anywhere in the world? How about a 100% command economy? You don't need perfect examples of things to observe what happens when economies closely approach those extremes. > How can you honesty determine if the federal reserve helps or hurts the economy? By observing how horrible the business cycle was prior to its existence.
      - 8y ago