8 ms·
How a New York Times Software Engineer Ended Up Covering Miss America
- danso 8y ago> There was no database, but the best databases aren’t handed to reporters. They’re handmade. I opened a blank spreadsheet and started digging through years of old news clips and pageant websites. I tracked who was competing where, what titles they were winning, and how. The more experienced I get in journalism and data work, the more convinced I am that the spreadsheet is the first and best tool for the kind of bespoke, flexible, and iterative data collection and modeling that journalists find themselves having to do -- i.e. building a dataset from scratch.
- CaliforniaKarl 8y agoI wonder how things would be different if HyperCard were still around.
- wavefunction 8y agoHyperCard was a fancier PowerPoint. I was deep into HyperCard at the relevant time.
- D-Coder 8y agoI'm using LiveCode for my one-time HyperCard stacks (http://livecode.com/ http://livecode.com/) (was called Runtime Revolution for a few years).
- colechristensen 8y agoI bet there is a space there for something between R and Excel. Something where everything is "as-code" in a scm friendly format but the primary interaction could be equally powerful in a text editor or a cell space. The "workbook" path of things like Jupityr is close.
- omaranto 8y agoI'd love a little spreadsheet widget for Jupyter that was only for data entry (i.e., a formula-less spreadsheet) and produced a Pandas dataframe.
- LolWolf 8y agoYou can do this with xlwings![0] Would highly recommend. —— [0] https://www.xlwings.org https://www.xlwings.org
- Bishonen88 8y agoI don't believe that's what the poster meant, which you're replying to. xlwings is a python library used to connect to excel files and e.g. change their contents programmatically. I don't think you can use it (out of the box anyway) to display and change data 'as-a-spreadsheet' in a jupyter notebook [e.g. by showing an iframe].
- LolWolf 8y agoYeah, unsure about what the parent may have intended. I use it as a data IO for stuff that I want to manipulate in Python somewhat-interactively, by having an Excel sheet open on the side as I work in Python.
- x1798DE 8y agoJupyter lab has a spreadsheet editor. I haven't actually used it so I'm not sure if you can calculate with it, but I'd be mildly surprised if you could.
- hsribei 8y agoI hear good things about spreadsheet functionality in org-mode.
- ycombinete 8y agoI use Orgmode. A lot. It's my primary organisation method, as the General Manager for a small 3 person sub-company, as well as my household organisation. I've always, unwittingly, organised myself via two systems: 1. Lists 2. Matrices I love the tables in Orgmode [0]. They are intuitive, simple, and just beautifully done. That said, the spreadsheet functionality is a different beast altogether. Even though it's the same interface as the tables, it adds an exponential layer of complexity, and separation from how most people view a spreadsheet. It's powerful, but a bit daunting, and I don't see many journalists being able to grok it. [0] https://www.youtube.com/watch?v=JHKrTsiz4JU&index=25&list=PLVtKhBrRV_ZkPnBtt_TD1Cs9PJlU0IIdE https://www.youtube.com/watch?v=JHKrTsiz4JU&index=25&list=PL...
- mgiannopoulos 8y agoI only discovered queries in Google Sheets two days ago and it was like magic / “why didn’t I know about this all these years” https://support.google.com/docs/answer/3093343?hl=en https://support.google.com/docs/answer/3093343?hl=en
- hyperman1 8y agoExcel has the nasty idea to modify what you type in the cell. If it seems date-like or number-like, it will convert the content, possibly corrupting interesting details in the process. You can of course counteract this with a quote or with formatting, but you have to remember to do this consistently. I understand why they do it for a spread sheet: It is optimized for calculating. But this tendency makes excel untrustworthy for general processing.
- bonniemuffin 8y agoThere's a gene called Septin-6, abbreviated Sept6. You can tell when gene expression data has been through an Excel cut/paste cycle, because Sept6 been converted to September 6. Oh Excel, you think you're so smart.
- tixocloud 8y agoMy last name consistently gets autocorrected from ‘teh’ to ‘the’ and my name tags for conferences get misprinted.
- weliketocode 8y agoSurprised to see so much support for google sheets over excel. Of course, all the aforementioned issues with excel can be avoided or changed in settings. Has the pendulum really swung so far to the other side that sheets is the consistently better option? What about keyboard shortcuts and power users?
- applecrazy 8y agoI believe Sheets does have keyboard shortcuts. And in response to your comment, Excel is still the de-facto tool for people in analyst and non-technical positions.
- badbug 8y agoFor some people, Google sheets was their first spreadsheet and people tend to stick with what they know.
- ggm 8y agoNosql and a jupyter workbook?
- CaliforniaKarl 8y agoThe methods may be different (using a search engine instead of a card catalog), and the locales might have changed (using an online data store at your desk instead of going to a Library), but one thing that hasn't changed is that lots of information doesn't exist in structured form.
- aaron695 8y agoSimilar story for Antony Green a famous Australian election analyst. https://en.wikipedia.org/wiki/Antony_Green https://en.wikipedia.org/wiki/Antony_Green
- rochefort 8y agoI'd have been more impressed if he'd managed to uncover her :-D
- ceph_ 8y agoKeep the low effort joke posts on Reddit. And if you had read the article first, you'd know the software engineer is a she.
- kgwgk 8y agoSubstituting uncover for cover is not even required to make a sexual joke: see acception 6a in https://www.merriam-webster.com/dictionary/cover https://www.merriam-webster.com/dictionary/cover
- wallflower 8y agoThis reminds me of one of my favorite stories. Mariel Padilla who was a student won a Pulitzer for her help in creating a database as an intern that was essential in keeping track of the 24/7 opioid crisis reporting that the Cincinnati Enquirer was doing. https://www.cjr.org/the_profile/mariel-padilla-pulizter-cincinnnati.php https://www.cjr.org/the_profile/mariel-padilla-pulizter-cinc... https://news.ycombinator.com/item?id=16891330 https://news.ycombinator.com/item?id=16891330
- Dangeranger 8y agoAs I remember she heard about winning the award during a journalism class. Quite a nice story, and she did outstanding work.
- catacombs 8y agoShe was part of a team that won the Pulitzer. She didn't win one herself.
- wallflower 8y agoI think of winning a Pulitzer in the non-individual categories like a film winning an Oscar in the non-individual categories. It is a grand achievement that was made possible by the efforts of many. A non-individual award at the Pulitzer level is still a remarkable achievement.
- deskamess 8y agoI don't know why, but I was kind of surprised when I got to the end. I was expecting more... some sort of conclusion. Other than that, yeah, use the best tools available and especially if you are getting data in an async manner, a cell-based direct edit tool like Excel is great.
- hn_throwaway_99 8y agoYeah, felt like a Richard Linklater film to me: "What, that's it? So you entered some data into Excel. And...?"
- catacombs 8y agoYou're not alone. The story the author is talking about -- https://www.nytimes.com/2018/09/10/style/miss-america-2019-pageant.html https://www.nytimes.com/2018/09/10/style/miss-america-2019-p... -- doesn't include much in terms of data analysis nor graphics to visualize it. The story was interesting but not one that is data-driven, like other Times pieces. The Insider essay was just hype: "Look at me! I'm a software engineer covering a beat! Go me!"
- senthil_rajasek 8y agoI watch sports a lot. A well executed shot or play is a delight to watch. Can someone explain why Miss America or a beauty pageant is interesting in 2018?
- badbug 8y agoBoobs
- deleted 8y ago[deleted]
- mdrzn 8y agoSo? Not to be rude, but I was expecting something more from this story than "I filled up a spreadsheet and talked to someone" :/ maybe I'm missing the point of this story?
- catacombs 8y agoNo, you're right. That is literally the entire point of the story. People on Twitter showered the author with praise. I mean, filled out a spreadsheet and talked to people, as you said, but there wasn't really that much besides that. In addition, the main story doesn't seem to include of her data.