3 ms·
As someone who has built event pipelines from GDELT data, I'd advise folks to not start (a GDELT project) with high expectations. Not only are there plenty of d
by arusahni 6y ago
As someone who has built event pipelines from GDELT data, I'd advise folks to not start (a GDELT project) with high expectations. Not only are there plenty of duplicates, but tagging quality is all over the place, especially for non-Western media.
- artembugara 6y agoI worked with GELT data for a long time. Same feedback. What are you using instead of GDELT now?
- arusahni 6y agoThat was with a previous org, so thankfully not something I have to worry about now. Our domain was pretty well defined, and we weren't building a real-time system, so we developed an adjudication tool and hired a few part time employees to evaluate and clean up our output.
- chris_f 6y agoThe one nice thing about GDELT is it translates non-english language news so that you can use an english search term and see articles in different language. It's not perfect (i.e. machine translation), but I am not aware of any other service that provides similar functionality. More info: https://blog.gdeltproject.org/gdelt-translingual-translating-the-planet/ https://blog.gdeltproject.org/gdelt-translingual-translating... (unrelated: scrolling on that page is horrible)
- prohobo 6y agoAgreed, it's very hit or miss in some situations. I'm hoping to keep evolving this to clean it up and eek out more salient information. I think GDELT is the best source, nothing else comes close to coverage in my opinion, it's just that all the articles it gathers revolve around world events, as opposed to all of the millions of topics of discussion and niche interests people post about.
- arusahni 6y agoI agree that GDELT is the best source. It's a hard problem, for sure. I just think that their project's mission and description may lead folks to believe it's this perfect data set, which it isn't.