7 ms·
A File Format Led to a Crossword Scandal (2019) [video]
- tzs 6y agoHere's the FiveThirtyEight article about this mentioned a few times in the video [1]. [1] https://fivethirtyeight.com/features/a-plagiarism-scandal-is-unfolding-in-the-crossword-world/ https://fivethirtyeight.com/features/a-plagiarism-scandal-is...
- ireflect 6y agoThere's a footnote about Saul's interesting name, which leads to: Pawanson is a bit quirky — his unusual last name is the product of a decision he made years ago. "I was born Paul Swanson," he said. "But I thought, 'there are lots of Paul Swansons out there. 'So I changed it." Amazing!
- servercobra 6y agoAs someone with a name completely unique in the history of the world (so far as I have found), there are certain advantages! I wouldn't be surprised if people do this more often in the future. It is pretty nice that if you Google my name, you only get results about me.
- matt-attack 6y agoInteresting. I relish the fact that when you google my name you get pages and pages of a semi-famous figure that, honestly, most people haven't heard of. I cherish the anonymity.
- busyant 6y agoI have a teacher friend named Mike Pence. He tells me that it's impossible for students to snoop on him because he is "google-proofed."
- mdonahoe 6y agoNice! I too was intrigued by his unusual name, and went on a quest to see if he had it changed from the more common "Paul Swanson". It's a very interesting choice to just swap the letters like that instead of going for a completely different name. It would be funny if his name gets included in a crossword puzzle, and people second guess the spelling.
- wolfhumble 6y agoI don't know anything about him or his decision to change from Paul to Saul, but Paul/Saul is on of the most important Christian apostles. As both Jew and Roman citizen, his Jewish name was Saul (from the Jewish king Saul in the Old Testament maybe?) and his Roman name was Paul. So just changing the first letter might not be completely random. :-) https://en.wikipedia.org/wiki/Paul_the_Apostle#Names https://en.wikipedia.org/wiki/Paul_the_Apostle#Names
- xorcist 6y agoThe question that directly pops up is why not Pwanson?
- xmprt 6y agoI didn't realize unix tools were this powerful. That's an amazing story.
- fiddlerwoaroof 6y agoYeah, Unix utilities and the whole text processing paradigm can do some amazing things if you know how to design for it. I’ve been doing some Cloudformation work recently, and it’s so easy just to throw together dashboards to watch the progress and outcome of a deploy.
- smabie 6y agoI think the point is that they're not, usually, this powerful. Saul made a deliberate choice to create a file format that would be extremely amenable to these tools.
- thaumasiotes 6y ago> Despite Parker’s denial, many in the crossword world see willful plagiarism in Parker’s puzzles, and they see the database that revealed the repetition as a tool of justice. “It’s like a murder mystery solved 50 years later with DNA evidence,” Matt Gaffney, a professional crossword constructor, told me. There's a response to postmodernism that says "reality is that which, when you ignore it, doesn't go away". I have a hard time seeing this as a "scandal"; it's firmly in the class of things that aren't a problem unless someone tells you you have a problem. A murder has victims. But if you're unhappy that the crossword puzzle you solved last week was secretly a rerun from 10 years ago, it's not obvious whether the blame for your unhappiness should go to the guy who reran the puzzle, or the guy who told you it was a rerun.
- labster 6y agoIf you’re unhappy after someone told you you worked a rerun crossword puzzle, maybe blame yourself? The only thing changing is your interpretation of your own experiences.
- thaumasiotes 6y agoI was confused by this response, because it appears to just repeat the content of my original comment. Now I'm more confused that my comment was upvoted and this was downvoted.
- labster 6y agoI'm guessing it's because you said it in an indirect way, and I said it directly. And people don't like being told that their gut feelings and outrage are only in their own head. I'm never really sure which is the right way to approach people -- the indirect approach goes over some people's heads sometimes (like me, a little this time) but the direct approach often gets outright rejected from confirmation bias. Teaching is hard, man.
- mkl 6y agoI don't think it's the rerunning that's the problem, but the misattribution, claiming others' work as their own or denying them credit (and presumably royalties).
- abalaji 6y agoThis is an awesome story, I especially like the speaker's organization of the narrative taking us along for the ride. Maybe this will be the push I need do a better job learning Unix tools.
- colmvp 6y agoThe delivery was very engaging and a good example to other engineers on how crafting a compelling narrative can help drive home the importance of your work.
- paxys 6y agoIt's interesting that while sophisticated plagiarism detecting software is commonplace for writing submissions at newspapers, book publishers, universities etc., they don't bother doing the same with crosswords (and probably other puzzles like Sudoku).
- rabidrat 6y agoHi, this is Saul. If you like this kind of simplicity-first data-exploration approach, you might want to check out another project of mine, VisiData (visidata.org). It's specifically for lightweight data exploration and analysis and it runs directly in the terminal.
- matthuggins 6y agoGood job selling yourself this time!
- manjalyc 6y agoHey, just wanted to say that this looks like a really cool and useful project. I work with a few medical databases and sometimes I just need a very quick breakdown of specific data and while I usually just write a short script, the utility and portability of this code looks fantastic to me. Which brings me to a question, how well does this program handle moderately large databases (~100GB-1TB) in your (or anyone else's) experience? In other words does it try to load everything into memory, or does it query as needed when given a database?
- rabidrat 6y agoIt loads everything into memory, so it works well with <1GB datasets. The architecture could be changed to allow for larger datasets like yours, but that would likely be a large undertaking and would probably be a paid feature.
- rolandog 6y agoHey Saul, your talk was great and engaging! Great work!
- smitty1e 6y ago> "It's not hoarding if it's organized." Oh, that's getting thugged.
- TheRealPomax 6y agoIf it's organized, now it's archiving.
- ericsoderstrom 6y agoWere the misattributed authors aware that they were being given credit for puzzles they didn't write? I'm assuming they must have been.
- rabidrat 6y agoThe misattributed authors are fake names, admitted by Timothy Parker himself. "Tim Burr" is one mentioned in the talk.
- rafaelturk 6y agoData is beautiful
- Erwin 6y agoThe author's biography is quite fascinating. If there's a museum of visualization, it'd be an exhibit: https://www.saul.pw/biograph/ https://www.saul.pw/biograph/