4 ms·
I just did a quick query grouping all sources and number of articles per source in the database (currently about 4 days worth of data): https://pastebin.com/Hyk
by prohobo 6y ago
I just did a quick query grouping all sources and number of articles per source in the database (currently about 4 days worth of data): https://pastebin.com/HyktFpML https://pastebin.com/HyktFpML
As you can see, there are already almost 8000 sources. I'm honestly not sure how GDELT scrapes its data though, but this is what they do for over 5 years now and it's backed by Google Cloud processing, so I'm sure it's ever expanding and very thorough.
The kinds of articles which are discovered revolve around world events, however. So you won't see random blog posts, or traffic reports. Instead you'll see geopolitics, big tech news, and cultural discourse.
Trending stories are found for the entire set of articles in a batch, with no separation for source, category, or location. If the top stories all happen to be in the USA in a given moment, then that's all you'll find. If NYT happens to publish an article that fits an overall story being talked about in the rest of the batch, it'll get added to that story.
Trending popularity is decided by how many individual sources talk about the given topic. This is good enough to identify the Zeitgeist of a moment, but it likely won't catch nuances in evolving stories.
- wombatmobile 6y agoThanks for compiling that list. I have had a quick look and downloaded it for a longer look. Some GDELT methodology is extolled here: https://blog.gdeltproject.org/gdelt-2-0-our-global-world-in-realtime/ https://blog.gdeltproject.org/gdelt-2-0-our-global-world-in-...