4 ms·

History as a giant data set: how analysing the past could help save the future

24 points by oska 7y ago

chkaloon 7y ago
The traditional historian arguments against this really show the lack of rigor inherent in the social sciences, or alternatively, a self-interest bias of some sort. Complaining that “humans can’t be reduced to numbers” completely misses how probabilistic analysis works. Anyone with basic statistical knowledge wouldn’t dismiss this out of hand unless someone’s ox is getting gored. I agree that it is a complex system with many variables involved, but humans aren’t magical. Their social systems likely follow patterns like any other complex system.
- hos234 7y ago
  Niall Ferguson has written an entire book on how Network Science/Graph theory concepts can be applied to historical events - The Square and the Tower. So it's not all bad. More like people are just waking up to possibilities.
angry_cactus 7y ago
Surprised the article doesn't mention Isaac Asimov's Foundation in which psychohistorians try to predict the future with mathematics, similar to Turchin's idea.
- nikbackm 7y ago
  Hm, trying to predict the weather is probably much easier.
ganzuul 7y ago
This is such a complex issue... On the most abstract level thermodynamic laws are defined in a closed space and those don't exist in nature. We would not have much history at all unless we existed in a space without divergence, tending to extremes instead of equilibrium. History isn't an accurate account of what has happened. There is a lot of ego involved and later an ego-nation complex. History is basically an account of biases and extremes instead of a model of human behavior. Perhapse we can make global laws to tend to equilibrium, designed with the fanciful use of AI and psychology/biology. Currently laws tend to extremes, motivated by profit.
- badrabbit 7y ago
  There is history and there are patterns derived from. Biased and incorrect history will likely be an outlier or it will form a weaker pattern. E.g.: communists win over the hearts of university students and rural folk. Then instability, coup/revolution and a controllable communist satellite is created. Is that true? Is any of it propaganda/fabrication? Which pattern is most common and can it be used to analyze Russia and China geopolitics against Ukraine ,US and UK?
  - paganel 7y ago
    Communists most definetely did not win the hearts of the rural folk in my country before coming to power with the Soviets’ help after WW2 (I live in Eastern Europe). The communists had relative success among the urban working classes and among some of the students/young folk (that was correct). But even among students, they mostly managed to attract what were perceived back then as “non-native enough” students, that is the Jewish students mostly (because the Jewish were let’s say more than ostracized back then and the communists were one of the few organizations who received them with no questions asked), but still, the great majority of the student class was still right-wing and nationalistic. All this to say that it is impossible to draw up generalizations (which most of them are made up based on Hollywood movies anyway).
    - badrabbit 7y ago
      I know a few cases where the rural class analogy is correct,including China. Any guerilla force will use that approach if the military is siding with the current government [1] , so the patter is only partially accurate,we can analyze when it is accurate and apply that lesson in the present. So,it has nothing to do with communist ideology, does that mean we can apply that to neo-nazis and other terror groups (al-qaeda is in there). This sort of analysis allows us to ask why patterns exist,the patterns are a starting point,not the conclusion. [1] https://en.m.wikipedia.org/wiki/List_of_guerrilla_movements https://en.m.wikipedia.org/wiki/List_of_guerrilla_movements
      - 7y ago
aaron695 7y ago
Not much difference between historians and fiction writers. Both tell a story of the world that's interesting, often insightful and totally made up. Post newspapers you might be able to find data, removing all the BS like what we know exists in today's newspapers.... but it'd be hard.
JoeAltmaier 7y ago
We have just one history. There's a real danger of over-training or over-fitting the data.

24 points by oska 7y ago

The traditional historian arguments against this really show the lack of rigor inherent in the social sciences, or alternatively, a self-interest bias of some sort. Complaining that “humans can’t be reduced to numbers” completely misses how probabilistic analysis works. Anyone with basic statistical knowledge wouldn’t dismiss this out of hand unless someone’s ox is getting gored. I agree that it is a complex system with many variables involved, but humans aren’t magical. Their social systems likely follow patterns like any other complex system.

hos234 7y ago
Niall Ferguson has written an entire book on how Network Science/Graph theory concepts can be applied to historical events - The Square and the Tower. So it's not all bad. More like people are just waking up to possibilities.

angry_cactus 7y ago

Surprised the article doesn't mention Isaac Asimov's Foundation in which psychohistorians try to predict the future with mathematics, similar to Turchin's idea.

nikbackm 7y ago
Hm, trying to predict the weather is probably much easier.

ganzuul 7y ago

This is such a complex issue... On the most abstract level thermodynamic laws are defined in a closed space and those don't exist in nature. We would not have much history at all unless we existed in a space without divergence, tending to extremes instead of equilibrium. History isn't an accurate account of what has happened. There is a lot of ego involved and later an ego-nation complex. History is basically an account of biases and extremes instead of a model of human behavior. Perhapse we can make global laws to tend to equilibrium, designed with the fanciful use of AI and psychology/biology. Currently laws tend to extremes, motivated by profit.

badrabbit 7y ago
There is history and there are patterns derived from. Biased and incorrect history will likely be an outlier or it will form a weaker pattern. E.g.: communists win over the hearts of university students and rural folk. Then instability, coup/revolution and a controllable communist satellite is created. Is that true? Is any of it propaganda/fabrication? Which pattern is most common and can it be used to analyze Russia and China geopolitics against Ukraine ,US and UK?
- paganel 7y ago
  Communists most definetely did not win the hearts of the rural folk in my country before coming to power with the Soviets’ help after WW2 (I live in Eastern Europe). The communists had relative success among the urban working classes and among some of the students/young folk (that was correct). But even among students, they mostly managed to attract what were perceived back then as “non-native enough” students, that is the Jewish students mostly (because the Jewish were let’s say more than ostracized back then and the communists were one of the few organizations who received them with no questions asked), but still, the great majority of the student class was still right-wing and nationalistic. All this to say that it is impossible to draw up generalizations (which most of them are made up based on Hollywood movies anyway).
  - badrabbit 7y ago
    I know a few cases where the rural class analogy is correct,including China. Any guerilla force will use that approach if the military is siding with the current government [1] , so the patter is only partially accurate,we can analyze when it is accurate and apply that lesson in the present. So,it has nothing to do with communist ideology, does that mean we can apply that to neo-nazis and other terror groups (al-qaeda is in there). This sort of analysis allows us to ask why patterns exist,the patterns are a starting point,not the conclusion. [1] https://en.m.wikipedia.org/wiki/List_of_guerrilla_movements https://en.m.wikipedia.org/wiki/List_of_guerrilla_movements
    - 7y ago

aaron695 7y ago

Not much difference between historians and fiction writers. Both tell a story of the world that's interesting, often insightful and totally made up. Post newspapers you might be able to find data, removing all the BS like what we know exists in today's newspapers.... but it'd be hard.

JoeAltmaier 7y ago

We have just one history. There's a real danger of over-training or over-fitting the data.