5 ms·
Where did they find the text samples? It strikes me that the corpus of informal text samples has skyrocketed in the past two decades. Remind me of how most hea
by darkerside 4y ago
Where did they find the text samples? It strikes me that the corpus of informal text samples has skyrocketed in the past two decades.
Remind me of how most heat maps are just population density maps.
- cycomanic 4y agoGoogle books it says so in the article
- antognini 4y agoIt's the Google N-grams dataset. It covers all the books that Google has scanned in their corpus, which, as I understand it, is virtually every book that's been published in the 20th century (at least for English). I agree that this might just be measuring increasing informality in published works.
- lumost 4y agoThis could also be a bias in the google books dataset. As I recall, the older google books dataset relied heavily on library preservation. It's probable that library curation would filter books for utility, longevity, and educational purpose. These filters would certainly penalize informal books.
- TimTheTinker 4y agoMy first thought as I scanned the abstract was that since the '90s, the volume of novels with informal dialog has gone through the roof -- especially novels for teens and young adults.
- permo-w 4y agothe amount of trashy celebrity autobiographies/books written by self-help gurus/informal-style recipe books being published has almost certainly risen in recent years too
- permo-w 4y ago>informal text informal and by less educated people. obviously there will be big exceptions, but I would imagine the % of computer-indexed text (i.e. books and newspapers) from between 1895 and 1995 that wasn't written by someone with a degree is not very high. similar to the way you can get a distorted view of the Romans (or nearly any literate historical society) because nearly (if not) all of their literature was written by the educated upper classes