10 ms·

The New York Times course to teach its reporters data skills is now open-source

427 points by espeed 7y ago

minimaxir 7y ago
I know Hacker News dislikes spreadsheets as opposed to statistical programming languages like R and Python, and there have been many startups trying to disrupt that paradigm, but time and time again they have been the most reliable tools for data analysis and collaborating with nontechnical users. Even as a data scientist working with other data scientists, my most common deliverable is a Google Sheet.
- bitlax 7y ago
  "All enterprise software competes with Excel. All productivity software competes with emailing things to yourself." https://twitter.com/pavelasamsonov/status/1106534188159778817?lang=en https://twitter.com/pavelasamsonov/status/110653418815977881...
  - scarejunba 7y ago
    All scheduling software competes with a pen and a piece of paper?
    - toomuchtodo 7y ago
      Or a notebook.
    - duxup 7y ago
      The old clocking in system with a card and stamp from the clock we're efficient and everyone understood how it worked.
      - dragonwriter 7y ago
        That's minimal time tracking, not scheduling. It's the difference between retrospective and prospective.
        duxup 7y ago
        Good point.
        hef19898 7y ago
        Kanban systems with real physical cards work pretty well. And you they are way more resilient than digital stuff.
    - reaperducer 7y ago
      All scheduling software competes with a pen and a piece of paper? According to the Wall Street Journal, yes. The Trendy New Way to Organize Your Schedule: A Paper Planner https://apple.news/AMN1W-MtTRCecQ-qIeFUQSQ https://apple.news/AMN1W-MtTRCecQ-qIeFUQSQ
    - dragonwriter 7y ago
      Scheduling is a subset of productivity, and, IME, not an exception to the rule about productivity software and Excel.
    - smsm42 7y ago
      And a corkboard, yes. Or maybe a big wall calendar.
  - NicoJuicy 7y ago
    Thah, I dogfeed http://handlr.sapico.me http://handlr.sapico.me for not emailing it to myselve. It seems to work :p
    - Terretta 7y ago
      Note, sapico.me (without www) is a bare Windows Server welcome page. Could redirect it to www so Google search bot isn’t confused about what’s at sapico.me. https://community.spiceworks.com/topic/711755-need-to-redirect-http-domain-com-to-http-www-domain-com https://community.spiceworks.com/topic/711755-need-to-redire...
      - NicoJuicy 7y ago
        Well, my start page isn't even mobile friendly. I don't actually bring in clients though the web page, don't need it actually. It's just there, a refactored version ( yekyll) is already in the works
  - bigiain 7y ago
    R and Python will not replace Excel until they "correctly" transform gene names into dates or floating point numbers... https://science.howstuffworks.com/innovation/scientific-experiments/excel-is-autocorrecting-scientific-research-thats-not-cool.htm https://science.howstuffworks.com/innovation/scientific-expe... How can we have reproducibility in scientific research if we don't all make the same errors?
    - asdff 7y ago
      Don't blame excel for people being ignorant on number formatting. It's front and center in the ribbon menu.
- steve19 7y ago
  I don't think the folks here dislike spreadsheets for working with data. I think the opposition is against spreadsheets being used to make mission critical but very brittle applications. Also anything is better than SPSS.
  - reubens 7y ago
    Agreed, and as an entry level program I think jamovi leads the pack https://www.jamovi.org/ https://www.jamovi.org/
    - steve19 7y ago
      That looks awesome. Thanks for sharing. I never use SPSS myself but occasionally have to look at other people's work in the app and I loathe it.
  - jerzyt 7y ago
    In my consulting career, I've seen major organizations track data in Excel, updating each month by creating a new set of columns. An absolute nightmare to handle. Of course every month the structure changed slightly, to make it more fun. I've also seen values stored as an #rgb only - no actual value in a cell.
    - hef19898 7y ago
      Nothing wrong with analyzing data in Excel. Nothing wrong with running limited operations in Excel if it is done by the same people and highly standardized. Tracking major operations in Excel, mostly outside the existing ERP which is happening more often than not in my opinion, is big no-no. Side note: I know people pasting screenshots from other Excel sheets into Excel sheets....
    - tyxodiwktis 7y ago
      To add to that, I have seen billions of dollars in EMD traded off an Excel model that took 30 minutes to run (come in to work, hit F9 to recalc / pull in updated market data) go get coffee, come back, send trades to trading desk
- mehrdadn 7y ago
  For the Excel pros here: how do you ever template anything in Excel? Like if you want the same formula for two different tables, do you just copy-paste? How do you keep them in sync? There's just so many things I want to do in Excel and it seems so limited in what it can do that I can't fathom how non-programmers end up using it so successfully.
  - amanzi 7y ago
    I have seen formulas entered as plain text in a cell, and then a macro used to update other cells with that formula. You can then change the formula in one place and run the macro to update it everywhere.
  - rahimnathwani 7y ago
    Most Excel users would copy paste a formula even _within_ the same table, and don't know that Excel's built-in 'Tables' allow you to automatically keep a formula consistent between rows. I'm not sure whether there's a good answer to your question If there is, I'm sure that most 'Excel pros' don't know it.
  - kvna 7y ago
    The best way to create reusable formulas in your workbooks is to add your table to the data model (which creates an in-memory tabular cube) and write measures using the DAX formula language. This has the added benefit that a single formula can be written to aggregate data at different levels, for example, a sum can be calculated over days, months, or years. This will only allow you to share formulas in a single workbook. The data model in Excel is powerful and under utilized.
    - mehrdadn 7y ago
      Wow I'd never heard of DAX. This sounds awesome. Thank you!
  - grosswait 7y ago
    I've seen locked sheets used as data source for constants and formulas with variables, the cells reference those with lookups and variables. I've come across a few spreadsheets that made me want to meet and learn something from the authors. Some have been very elaborate and surprisingly resilient.
- duxup 7y ago
  I've seen folks in excel say "I'm not a programmer, so I did this in excel." and ... I was pretty damn impressed. Some of them had some of the logical thinking basics down to be pretty good at programming, if they gave it a try I suspect some might be damn good.
  - jumelles 7y ago
    No one's ever told them that they ARE programming. Just in Excel.
- cljs-js-eval 7y ago
  I don't dislike people using the statistical tools available to them, but in my own field (social sciences) there's a huge replication crisis going on right now. And a lot of that is due to people who were never good at math taking easy-to-use statistical tools like Excel and SPSS and blindly running stats without programming or math training. Is it too much to ask that people treat a field with a bit of respect? Like, just because NYT reporters can use some of these "data skills", can they hold off a bit until we figure out if they're even any good at statistical analysis after their crash course? We currently have an entire academic field that has to throw away a lot of their findings because tools like sheets and SPSS gave them false confidence. I don't have any higher hopes for the NYT newsroom.
  - filmgirlcw 7y ago
    I think that’s unfairly dismissive to the data scientists, statisticians, analysts and engineers who work at the New York Times and other major publications (as well as smaller, but crucially important places like Pro Publica). The purpose of this material isn’t to suddenly turn normal reporters into data scientists, it’s to give them a better grasp and understanding how how to evaluate different types of information that become important when reporting. I don’t know how good or bad this material is — a cursory glance shows that it’s very low-level, the type of stuff I learned in my 100 level accounting and stats classes as an undergrad. But I won’t dismiss this material being made available and potentially augmented for all — tho I wish it was stored in GitHub or GitLab. If you look through the material, there is nothing that actually says that someone who goes through this training will be a skilled data journalist. But it might just prevent poorly-interpreted articles like this [1] from being written. And for the record, I’ve worked with data journalists who were more skilled in math and computer science than the engineers I work with at giant tech companies. [1]: https://www.nytimes.com/2019/06/09/business/media/google-news-industry-antitrust.html https://www.nytimes.com/2019/06/09/business/media/google-new...
- lapnitnelav 7y ago
  Excel is an excellent tool, the most democratic data tool out there (and for the foreseeable short term future). Whether you are a finance person working on balancing the sheets (or whatever else they do, that's beyond my knowledge) or an operation person building complex macros, it is both versatile, easy to use and yet powerful. The byproducts of this is that it is also subverted by the genius of human ingenuity and you end up with some pretty interesting, awe-inspiring contraptions. Such as building a calendar system but using arrow graphs to fill it out. And that's fine, except when you try to scale things up to automate the process and save people money. Now you have to do some pretty wasteful engineering to accommodate this pesky creativity we have. That is the really interesting and a testament to being a really good software (in features and reach). It breaks the boundaries of the limited vocabulary of computers and therefore can only be fully leveraged by humans. It's both a great reminder that most people in the world are not on the same level as the crowd around here [1] and that as the group who create tools used by such an audience, we have to be mindful of that. On a positive note, I think there are some interesting work being done to rethink how to approach those tasks, I remember using one product in particular that had the right mix of being visually engaging while enforcing boundaries. But this won't solve the issue of users having to hack their way into getting what they want how they want it. [1] https://www.nngroup.com/articles/computer-skill-levels/ https://www.nngroup.com/articles/computer-skill-levels/
- m3kw9 7y ago
  Always should use higher level software(Excel) till you cannoto.
- patrick5415 7y ago
  I’m not really very facile with excel, but one place it’s really handy for me is situations that combine manual data entry with computation. A super simple example is keeping track of monthly bills when you have flat mates, and need to split them every month. It’s not so critical that I feel the need to version control it etc, but it’s still nice to have a visually inspectable record. Even though I spend most of my days writing code, anything I can dream up that involves python or whatever just seems unnecessarily opaque and baroque. A spreadsheet is ideally suited to the task.
  - deleted 7y ago
    [deleted]
- smsm42 7y ago
  I don't know who dislikes spreadsheets, I am a programmer by trade and I regularly use spreadsheets to accomplish dozens of tasks, from managing household budget to making semi-automated expense reports to keeping scores in local gaming club. Spreadsheets are an excellent tool if used properly, and more you learn about them more uses they reveal.
  - carlmr 7y ago
    Until you see motor simulations running in Excel which break in many ways. If your data can be manually entered it's decent, or if you just need to filter CSV by some columns. But anything above that please choose a proper programming language and data format.
    - smsm42 7y ago
      So you're saying you should use appropriate tools for the job? Fascinating idea!
bcks 7y ago
Off topic maybe: are there resources out there for teaching coders how to journalism?
- dlivingston 7y ago
  Outside of personal blogging, in what context would a programmer need to know the fundamentals of journalism?
  - AnimalMuppet 7y ago
    When running a startup, and you either want the press to talk to you, or the press decides that you're an interesting story. Larger companies have PR people to handle that kind of thing. In a startup, though, it could easily be a programmer who talks to the press, because there isn't anyone else.
    - mr_toad 7y ago
      There are courses on how to speak to the media. They are aimed at CEOs, politicians etc, or anyone who might need to speak to the media. Often taught by reporters, or ex-reporters. I’ve done one, I’d recommend it. I have rarely talked to journalists, but the skills are useful in any situation where you’re being ‘grilled’, like a job interview.
      - mehrdadn 7y ago
        That's cool. Where are these kinds of courses generally taught?
        mr_toad 7y ago
        I’d google for something like “media training for executives”. You can probably get courses in most major cities.
        7y ago
s4n1ty 7y ago
Perhaps they could teach them journalism instead.
- tomcam 7y ago
  BOOM! Allow me to upvote quickly because it will be flagged to oblivion. Well done, by new friend.
Naac 7y ago
Are they using the term open source correctly here? It's already muddled between open source, open core, and Free software ( RMS approved ). Couldn't they just day the course materials are now available?
- rrishi 7y ago
  I feel that courses co-opting the term "open source" is just unnecessary. Seems like a PR gimmick.
  - cyborgx7 7y ago
    The NYT never calls it "open source". That is an invention of this article.
- ivan_ah 7y ago
  Yeah I didn't see any mention of licensing. Is this CC BY-NC? CC0? Your guess is my guess. Context: I work in the OER space and I'm interested in the material, but can only use it if there is an explicit open license attached to it.
buttholesurfer 7y ago
Please stop "teaching" those idiots how to "code".
rahimnathwani 7y ago
I found it both amusing and annoying that the first line of the article mentions VLOOKUP. In my experience, there's never a good reason to use VLOOKUP. You can always achieve the same functionality using INDEX (in conjunction with MATCH). Using VLOOKUP means that your formulae break as soon as someone inserts a new column in the middle of your table. And clicking the cell with the formula doesn't immediately show you which two columns are used by the function. More (opinionated) Excel tips here: https://www.encona.com/posts/excel-best-practices https://www.encona.com/posts/excel-best-practices
- samdfonseca 7y ago
  So you just decided you were gonna find something to criticize before you even clicked the link, eh?
  - rahimnathwani 7y ago
    Yup. Without clicking the link I correctly guessed that the first sentence mentioned VLOOKUP /s
- drej 7y ago
  That is unless you parametrise that column index as well - you can do a MATCH to retrieve your desired column number and safeguard it against column insertions/deletions. (It weakens the case for VLOOKUP, but being quite a light Excel user, this is usually how I do things.)
3xblah 7y ago
"... keeping track of the 3,472,382 people currently running for the Democratic nomination for president." Is that a joke? https://ballotpedia.org/List_of_registered_2020_presidential_candidates https://ballotpedia.org/List_of_registered_2020_presidential...
- untog 7y ago
  ...yes?
maccard 7y ago
Any reason this links to a non-original source, with one of those annoying full screen email grab modals, when the article contains a link to the original release [0] ? [0] https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spreadsheets-adc43a93b919 https://open.nytimes.com/how-we-helped-our-reporters-learn-t...
- sctb 7y ago
  OK! We've updated the link from https://www.niemanlab.org/2019/06/the-new-york-times-has-a-course-to-teach-its-reporters-data-skills-and-now-theyve-open-sourced-it/ https://www.niemanlab.org/2019/06/the-new-york-times-has-a-c....
  - ionescu77 7y ago
    Thanks for keeping the link and posting here!
- ionescu77 7y ago
  Maybe because medium.com decided I've read to many articles this month. I currently have a couple of digital subscriptions (man they are adding up quite fast) I did not start paying for medium too.
rambojazz 7y ago
Can somebody please explain what "open source" means here? I could only find a bunch of files on a google drive, none of which have a licensing note?
- cyborgx7 7y ago
  "Open source" is what this sub-blogspam level article chose to call it. The NYT called it "Releasing our materials". https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spreadsheets-adc43a93b919 https://open.nytimes.com/how-we-helped-our-reporters-learn-t...
stakhanov 7y ago
Attempts to get journalists more up-to-speed with this sort of stuff are to be applauded. But the real problem is journalists (and their audiences) who, for a lack of professional ethics, don't give a crap about which parts of their stories can/cannot be backed up quantitatively. Besides selling newspapers, not giving a crap also has the great benefit that now they don't have to learn math, or programming, or logical thinking, or any of that.
westurner 7y ago
It's more work to verify all formulas that reference unnamed variables in a spreadsheet than to review the code inputs and outputs in a notebook. "Teaching Pandas and Jupyter to Northwestern journalism students" [in DC] https://www.californiacivicdata.org/2017/06/07/dc-python-notebook/ https://www.californiacivicdata.org/2017/06/07/dc-python-not... > http://www.firstpythonnotebook.org/ http://www.firstpythonnotebook.org/ You can also develop d3.js visualizations — just like NYT — with jupyter notebooks and whichever language(s). "Data-Driven Journalism" ("ddj") https://en.wikipedia.org/wiki/Data-driven_journalism https://en.wikipedia.org/wiki/Data-driven_journalism http://datadrivenjournalism.net/ http://datadrivenjournalism.net/ "The Data Journalism Handbook 1" https://datajournalism.com/read/handbook/one https://datajournalism.com/read/handbook/one "The Data Journalism Handbook 2" https://datajournalism.com/read/handbook/two https://datajournalism.com/read/handbook/two While there are a number of ScholarlyArticle journals that can publish notebooks, I'm not aware of any newspapers that are prepared to publish notebooks as NewsArticles. It's pretty easy to `jupyter convert --to html` and `--to markdown` or just 'Save as' Regarding expressing facts as verifiable claims with structured data in HTML and/or blockchains: "Fact Checks" https://news.ycombinator.com/item?id=15529140 https://news.ycombinator.com/item?id=15529140 Does this course recommend linking to every source dataset and/or including full citations (with DOI) in the article? Does this course recommend getting a free DOI for the published revision of an e.g. GitHub project repository (containing data, and notebooks and/or the article text) with Zenodo?
subpixel 7y ago
To be clear, these are supporting course materials (syllabus, data sets, cheat sheets, etc.), not actual instruction materials. I'd love to take the course.

427 points by espeed 7y ago

I know Hacker News dislikes spreadsheets as opposed to statistical programming languages like R and Python, and there have been many startups trying to disrupt that paradigm, but time and time again they have been the most reliable tools for data analysis and collaborating with nontechnical users. Even as a data scientist working with other data scientists, my most common deliverable is a Google Sheet.

bitlax 7y ago
"All enterprise software competes with Excel. All productivity software competes with emailing things to yourself." https://twitter.com/pavelasamsonov/status/1106534188159778817?lang=en https://twitter.com/pavelasamsonov/status/110653418815977881...
- scarejunba 7y ago
  All scheduling software competes with a pen and a piece of paper?
  - toomuchtodo 7y ago
    Or a notebook.
  - duxup 7y ago
    The old clocking in system with a card and stamp from the clock we're efficient and everyone understood how it worked.
    - dragonwriter 7y ago
      That's minimal time tracking, not scheduling. It's the difference between retrospective and prospective.
      - duxup 7y ago
        Good point.
      - hef19898 7y ago
        Kanban systems with real physical cards work pretty well. And you they are way more resilient than digital stuff.
  - reaperducer 7y ago
    All scheduling software competes with a pen and a piece of paper? According to the Wall Street Journal, yes. The Trendy New Way to Organize Your Schedule: A Paper Planner https://apple.news/AMN1W-MtTRCecQ-qIeFUQSQ https://apple.news/AMN1W-MtTRCecQ-qIeFUQSQ
  - dragonwriter 7y ago
    Scheduling is a subset of productivity, and, IME, not an exception to the rule about productivity software and Excel.
  - smsm42 7y ago
    And a corkboard, yes. Or maybe a big wall calendar.
- NicoJuicy 7y ago
  Thah, I dogfeed http://handlr.sapico.me http://handlr.sapico.me for not emailing it to myselve. It seems to work :p
  - Terretta 7y ago
    Note, sapico.me (without www) is a bare Windows Server welcome page. Could redirect it to www so Google search bot isn’t confused about what’s at sapico.me. https://community.spiceworks.com/topic/711755-need-to-redirect-http-domain-com-to-http-www-domain-com https://community.spiceworks.com/topic/711755-need-to-redire...
    - NicoJuicy 7y ago
      Well, my start page isn't even mobile friendly. I don't actually bring in clients though the web page, don't need it actually. It's just there, a refactored version ( yekyll) is already in the works
- bigiain 7y ago
  R and Python will not replace Excel until they "correctly" transform gene names into dates or floating point numbers... https://science.howstuffworks.com/innovation/scientific-experiments/excel-is-autocorrecting-scientific-research-thats-not-cool.htm https://science.howstuffworks.com/innovation/scientific-expe... How can we have reproducibility in scientific research if we don't all make the same errors?
  - asdff 7y ago
    Don't blame excel for people being ignorant on number formatting. It's front and center in the ribbon menu.
steve19 7y ago
I don't think the folks here dislike spreadsheets for working with data. I think the opposition is against spreadsheets being used to make mission critical but very brittle applications. Also anything is better than SPSS.
- reubens 7y ago
  Agreed, and as an entry level program I think jamovi leads the pack https://www.jamovi.org/ https://www.jamovi.org/
  - steve19 7y ago
    That looks awesome. Thanks for sharing. I never use SPSS myself but occasionally have to look at other people's work in the app and I loathe it.
- jerzyt 7y ago
  In my consulting career, I've seen major organizations track data in Excel, updating each month by creating a new set of columns. An absolute nightmare to handle. Of course every month the structure changed slightly, to make it more fun. I've also seen values stored as an #rgb only - no actual value in a cell.
  - hef19898 7y ago
    Nothing wrong with analyzing data in Excel. Nothing wrong with running limited operations in Excel if it is done by the same people and highly standardized. Tracking major operations in Excel, mostly outside the existing ERP which is happening more often than not in my opinion, is big no-no. Side note: I know people pasting screenshots from other Excel sheets into Excel sheets....
  - tyxodiwktis 7y ago
    To add to that, I have seen billions of dollars in EMD traded off an Excel model that took 30 minutes to run (come in to work, hit F9 to recalc / pull in updated market data) go get coffee, come back, send trades to trading desk
mehrdadn 7y ago
For the Excel pros here: how do you ever template anything in Excel? Like if you want the same formula for two different tables, do you just copy-paste? How do you keep them in sync? There's just so many things I want to do in Excel and it seems so limited in what it can do that I can't fathom how non-programmers end up using it so successfully.
- amanzi 7y ago
  I have seen formulas entered as plain text in a cell, and then a macro used to update other cells with that formula. You can then change the formula in one place and run the macro to update it everywhere.
- rahimnathwani 7y ago
  Most Excel users would copy paste a formula even _within_ the same table, and don't know that Excel's built-in 'Tables' allow you to automatically keep a formula consistent between rows. I'm not sure whether there's a good answer to your question If there is, I'm sure that most 'Excel pros' don't know it.
- kvna 7y ago
  The best way to create reusable formulas in your workbooks is to add your table to the data model (which creates an in-memory tabular cube) and write measures using the DAX formula language. This has the added benefit that a single formula can be written to aggregate data at different levels, for example, a sum can be calculated over days, months, or years. This will only allow you to share formulas in a single workbook. The data model in Excel is powerful and under utilized.
  - mehrdadn 7y ago
    Wow I'd never heard of DAX. This sounds awesome. Thank you!
- grosswait 7y ago
  I've seen locked sheets used as data source for constants and formulas with variables, the cells reference those with lookups and variables. I've come across a few spreadsheets that made me want to meet and learn something from the authors. Some have been very elaborate and surprisingly resilient.
duxup 7y ago
I've seen folks in excel say "I'm not a programmer, so I did this in excel." and ... I was pretty damn impressed. Some of them had some of the logical thinking basics down to be pretty good at programming, if they gave it a try I suspect some might be damn good.
- jumelles 7y ago
  No one's ever told them that they ARE programming. Just in Excel.
cljs-js-eval 7y ago
I don't dislike people using the statistical tools available to them, but in my own field (social sciences) there's a huge replication crisis going on right now. And a lot of that is due to people who were never good at math taking easy-to-use statistical tools like Excel and SPSS and blindly running stats without programming or math training. Is it too much to ask that people treat a field with a bit of respect? Like, just because NYT reporters can use some of these "data skills", can they hold off a bit until we figure out if they're even any good at statistical analysis after their crash course? We currently have an entire academic field that has to throw away a lot of their findings because tools like sheets and SPSS gave them false confidence. I don't have any higher hopes for the NYT newsroom.
- filmgirlcw 7y ago
  I think that’s unfairly dismissive to the data scientists, statisticians, analysts and engineers who work at the New York Times and other major publications (as well as smaller, but crucially important places like Pro Publica). The purpose of this material isn’t to suddenly turn normal reporters into data scientists, it’s to give them a better grasp and understanding how how to evaluate different types of information that become important when reporting. I don’t know how good or bad this material is — a cursory glance shows that it’s very low-level, the type of stuff I learned in my 100 level accounting and stats classes as an undergrad. But I won’t dismiss this material being made available and potentially augmented for all — tho I wish it was stored in GitHub or GitLab. If you look through the material, there is nothing that actually says that someone who goes through this training will be a skilled data journalist. But it might just prevent poorly-interpreted articles like this [1] from being written. And for the record, I’ve worked with data journalists who were more skilled in math and computer science than the engineers I work with at giant tech companies. [1]: https://www.nytimes.com/2019/06/09/business/media/google-news-industry-antitrust.html https://www.nytimes.com/2019/06/09/business/media/google-new...
lapnitnelav 7y ago
Excel is an excellent tool, the most democratic data tool out there (and for the foreseeable short term future). Whether you are a finance person working on balancing the sheets (or whatever else they do, that's beyond my knowledge) or an operation person building complex macros, it is both versatile, easy to use and yet powerful. The byproducts of this is that it is also subverted by the genius of human ingenuity and you end up with some pretty interesting, awe-inspiring contraptions. Such as building a calendar system but using arrow graphs to fill it out. And that's fine, except when you try to scale things up to automate the process and save people money. Now you have to do some pretty wasteful engineering to accommodate this pesky creativity we have. That is the really interesting and a testament to being a really good software (in features and reach). It breaks the boundaries of the limited vocabulary of computers and therefore can only be fully leveraged by humans. It's both a great reminder that most people in the world are not on the same level as the crowd around here [1] and that as the group who create tools used by such an audience, we have to be mindful of that. On a positive note, I think there are some interesting work being done to rethink how to approach those tasks, I remember using one product in particular that had the right mix of being visually engaging while enforcing boundaries. But this won't solve the issue of users having to hack their way into getting what they want how they want it. [1] https://www.nngroup.com/articles/computer-skill-levels/ https://www.nngroup.com/articles/computer-skill-levels/
m3kw9 7y ago
Always should use higher level software(Excel) till you cannoto.
patrick5415 7y ago
I’m not really very facile with excel, but one place it’s really handy for me is situations that combine manual data entry with computation. A super simple example is keeping track of monthly bills when you have flat mates, and need to split them every month. It’s not so critical that I feel the need to version control it etc, but it’s still nice to have a visually inspectable record. Even though I spend most of my days writing code, anything I can dream up that involves python or whatever just seems unnecessarily opaque and baroque. A spreadsheet is ideally suited to the task.
- deleted 7y ago
  [deleted]
smsm42 7y ago
I don't know who dislikes spreadsheets, I am a programmer by trade and I regularly use spreadsheets to accomplish dozens of tasks, from managing household budget to making semi-automated expense reports to keeping scores in local gaming club. Spreadsheets are an excellent tool if used properly, and more you learn about them more uses they reveal.
- carlmr 7y ago
  Until you see motor simulations running in Excel which break in many ways. If your data can be manually entered it's decent, or if you just need to filter CSV by some columns. But anything above that please choose a proper programming language and data format.
  - smsm42 7y ago
    So you're saying you should use appropriate tools for the job? Fascinating idea!

bcks 7y ago

Off topic maybe: are there resources out there for teaching coders how to journalism?

dlivingston 7y ago
Outside of personal blogging, in what context would a programmer need to know the fundamentals of journalism?
- AnimalMuppet 7y ago
  When running a startup, and you either want the press to talk to you, or the press decides that you're an interesting story. Larger companies have PR people to handle that kind of thing. In a startup, though, it could easily be a programmer who talks to the press, because there isn't anyone else.
  - mr_toad 7y ago
    There are courses on how to speak to the media. They are aimed at CEOs, politicians etc, or anyone who might need to speak to the media. Often taught by reporters, or ex-reporters. I’ve done one, I’d recommend it. I have rarely talked to journalists, but the skills are useful in any situation where you’re being ‘grilled’, like a job interview.
    - mehrdadn 7y ago
      That's cool. Where are these kinds of courses generally taught?
      - mr_toad 7y ago
        I’d google for something like “media training for executives”. You can probably get courses in most major cities.
        7y ago

s4n1ty 7y ago

Perhaps they could teach them journalism instead.

tomcam 7y ago
BOOM! Allow me to upvote quickly because it will be flagged to oblivion. Well done, by new friend.

Naac 7y ago

Are they using the term open source correctly here? It's already muddled between open source, open core, and Free software ( RMS approved ). Couldn't they just day the course materials are now available?

rrishi 7y ago
I feel that courses co-opting the term "open source" is just unnecessary. Seems like a PR gimmick.
- cyborgx7 7y ago
  The NYT never calls it "open source". That is an invention of this article.
ivan_ah 7y ago
Yeah I didn't see any mention of licensing. Is this CC BY-NC? CC0? Your guess is my guess. Context: I work in the OER space and I'm interested in the material, but can only use it if there is an explicit open license attached to it.

buttholesurfer 7y ago

Please stop "teaching" those idiots how to "code".

rahimnathwani 7y ago

I found it both amusing and annoying that the first line of the article mentions VLOOKUP. In my experience, there's never a good reason to use VLOOKUP. You can always achieve the same functionality using INDEX (in conjunction with MATCH). Using VLOOKUP means that your formulae break as soon as someone inserts a new column in the middle of your table. And clicking the cell with the formula doesn't immediately show you which two columns are used by the function. More (opinionated) Excel tips here: https://www.encona.com/posts/excel-best-practices https://www.encona.com/posts/excel-best-practices

samdfonseca 7y ago
So you just decided you were gonna find something to criticize before you even clicked the link, eh?
- rahimnathwani 7y ago
  Yup. Without clicking the link I correctly guessed that the first sentence mentioned VLOOKUP /s
drej 7y ago
That is unless you parametrise that column index as well - you can do a MATCH to retrieve your desired column number and safeguard it against column insertions/deletions. (It weakens the case for VLOOKUP, but being quite a light Excel user, this is usually how I do things.)

3xblah 7y ago

"... keeping track of the 3,472,382 people currently running for the Democratic nomination for president." Is that a joke? https://ballotpedia.org/List_of_registered_2020_presidential_candidates https://ballotpedia.org/List_of_registered_2020_presidential...

untog 7y ago
...yes?

maccard 7y ago

Any reason this links to a non-original source, with one of those annoying full screen email grab modals, when the article contains a link to the original release [0] ? [0] https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spreadsheets-adc43a93b919 https://open.nytimes.com/how-we-helped-our-reporters-learn-t...

sctb 7y ago
OK! We've updated the link from https://www.niemanlab.org/2019/06/the-new-york-times-has-a-course-to-teach-its-reporters-data-skills-and-now-theyve-open-sourced-it/ https://www.niemanlab.org/2019/06/the-new-york-times-has-a-c....
- ionescu77 7y ago
  Thanks for keeping the link and posting here!
ionescu77 7y ago
Maybe because medium.com decided I've read to many articles this month. I currently have a couple of digital subscriptions (man they are adding up quite fast) I did not start paying for medium too.

rambojazz 7y ago

Can somebody please explain what "open source" means here? I could only find a bunch of files on a google drive, none of which have a licensing note?

cyborgx7 7y ago
"Open source" is what this sub-blogspam level article chose to call it. The NYT called it "Releasing our materials". https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spreadsheets-adc43a93b919 https://open.nytimes.com/how-we-helped-our-reporters-learn-t...

stakhanov 7y ago

Attempts to get journalists more up-to-speed with this sort of stuff are to be applauded. But the real problem is journalists (and their audiences) who, for a lack of professional ethics, don't give a crap about which parts of their stories can/cannot be backed up quantitatively. Besides selling newspapers, not giving a crap also has the great benefit that now they don't have to learn math, or programming, or logical thinking, or any of that.

westurner 7y ago

It's more work to verify all formulas that reference unnamed variables in a spreadsheet than to review the code inputs and outputs in a notebook. "Teaching Pandas and Jupyter to Northwestern journalism students" [in DC] https://www.californiacivicdata.org/2017/06/07/dc-python-notebook/ https://www.californiacivicdata.org/2017/06/07/dc-python-not... > http://www.firstpythonnotebook.org/ http://www.firstpythonnotebook.org/ You can also develop d3.js visualizations — just like NYT — with jupyter notebooks and whichever language(s). "Data-Driven Journalism" ("ddj") https://en.wikipedia.org/wiki/Data-driven_journalism https://en.wikipedia.org/wiki/Data-driven_journalism http://datadrivenjournalism.net/ http://datadrivenjournalism.net/ "The Data Journalism Handbook 1" https://datajournalism.com/read/handbook/one https://datajournalism.com/read/handbook/one "The Data Journalism Handbook 2" https://datajournalism.com/read/handbook/two https://datajournalism.com/read/handbook/two While there are a number of ScholarlyArticle journals that can publish notebooks, I'm not aware of any newspapers that are prepared to publish notebooks as NewsArticles. It's pretty easy to `jupyter convert --to html` and `--to markdown` or just 'Save as' Regarding expressing facts as verifiable claims with structured data in HTML and/or blockchains: "Fact Checks" https://news.ycombinator.com/item?id=15529140 https://news.ycombinator.com/item?id=15529140 Does this course recommend linking to every source dataset and/or including full citations (with DOI) in the article? Does this course recommend getting a free DOI for the published revision of an e.g. GitHub project repository (containing data, and notebooks and/or the article text) with Zenodo?

subpixel 7y ago

To be clear, these are supporting course materials (syllabus, data sets, cheat sheets, etc.), not actual instruction materials. I'd love to take the course.