5 ms·

This is a really naive search method for a person whose job is to be good at this. It's like watching the Spotlight team going through volumes of data themselv

by fenollp 11y ago

This is a really naive search method for a person whose job is to be good at this. It's like watching the Spotlight team going through volumes of data themselves… Isn't Data Journalist a thing already?

ethanbond 11y ago
Oh good. Another Data X that we can debate the meaning of for the next couple years.
jldugger 11y ago
What's your argument exactly? That they should have used narrower terms until they found an article that was already written?
- fenollp 11y ago
  They use a syntax-aware search (thus refining for ME, MY, …). I thought (hoped) that journalists among all people would be the best at semantic search. Here the guy is just saying "whoa great dude and next I'll use regexps oh my god woaw". We have word2vec, doc2vec, many cores, … why aren't journalists using the tools they should be using?
  - fudged71 11y ago
    I would read this post if you wrote it... because I don't know what word2vec & doc2vec are and why they are useful. If you write the post maybe someone will be inspired to build a tool for journalists using these algos.
  - matnewton 11y ago
    That's an interesting question - if you read the stories of how Edward Snowden's story came to light, you'll find that his main issue with Glenn Greenwald was to get the guy to use technology properly. Hopefully out of the many tech heads who read HN, one will have the same thought as you and build the tool.
  - seanccox 11y ago
    None of the journalists I have met or worked with in six years in publishing and investigative due diligence know about the tools you are proposing. I recognize regex, because of a problem I needed to solve (that later became two problems), but I would love to learn more about the others. If you would consider writing that up, I would happily edit it for you.
awl130 11y ago
i disagree. i don't think the author's search method is intuitive at all. i think, even for a journalist, it's normal behavior to search by proper nouns and words of specificity, rather than pronouns and what we normally deem "filler" words such as "me" and "my".
rdancer 11y ago
Their job isn't to be good. Their job is to convince the masses that they are. In order to sell that idea, it is beneficial if they believe their own bullshit, and that selects people who are not very good.

by fenollp 11y ago

ethanbond 11y ago

Oh good. Another Data X that we can debate the meaning of for the next couple years.

jldugger 11y ago

What's your argument exactly? That they should have used narrower terms until they found an article that was already written?

fenollp 11y ago
They use a syntax-aware search (thus refining for ME, MY, …). I thought (hoped) that journalists among all people would be the best at semantic search. Here the guy is just saying "whoa great dude and next I'll use regexps oh my god woaw". We have word2vec, doc2vec, many cores, … why aren't journalists using the tools they should be using?
- fudged71 11y ago
  I would read this post if you wrote it... because I don't know what word2vec & doc2vec are and why they are useful. If you write the post maybe someone will be inspired to build a tool for journalists using these algos.
- matnewton 11y ago
  That's an interesting question - if you read the stories of how Edward Snowden's story came to light, you'll find that his main issue with Glenn Greenwald was to get the guy to use technology properly. Hopefully out of the many tech heads who read HN, one will have the same thought as you and build the tool.
- seanccox 11y ago
  None of the journalists I have met or worked with in six years in publishing and investigative due diligence know about the tools you are proposing. I recognize regex, because of a problem I needed to solve (that later became two problems), but I would love to learn more about the others. If you would consider writing that up, I would happily edit it for you.

awl130 11y ago

i disagree. i don't think the author's search method is intuitive at all. i think, even for a journalist, it's normal behavior to search by proper nouns and words of specificity, rather than pronouns and what we normally deem "filler" words such as "me" and "my".

rdancer 11y ago

Their job isn't to be good. Their job is to convince the masses that they are. In order to sell that idea, it is beneficial if they believe their own bullshit, and that selects people who are not very good.