2 ms·

Every dictionary in every language (including related things such as Glossaries and dictionaries related to specific subjects, fields, genres, etc., i.e., Law D

Every dictionary in every language (including related things such as Glossaries and dictionaries related to specific subjects, fields, genres, etc., i.e., Law Dictionaries, Thieves' Cant dictionaries (hey, maybe they are one in the same! I'm just kidding... well, mostly!) should be collected and subsequently turned into simple formatted text files (CSV, JSON, "Word"="Definition Of Word", ?), one per dictionary. Each dictionary file should have a meta-data header (also with some common format) -- which describes the name of the dictionary, the year, the language, the subject, the country of origin -- and anything else that might be of relevance or use. These files should then be publicly available -- like on GitHub or what-have-you -- for AI training ("AI, read me this ancient document but only use these ancient dictionaries (or "only use dictionaries before year: <Year Long Ago>") when you translate..."). Maybe we'd also want the AI to generate a yearly snapshot (a dictionary in its own right) of all of the words (and phrases) that it knows about in a given year. This way, when new phrases/words come into popular usage, AI's can identify when/where and how (approximately) the word came into popular usage... Also, same thing, but with word meanings... Words tend to shift their meanings around as great amounts of time pass... When that happens -- the past (that is, all past/historical writings in that language) are effectively lost because the meanings are lost, and now those writings read as gibberish... Languages change and shift over time! We could preserve more of Earth's history (as historical writings whose meaning and intent could be properly deciphered in the future) if we can preserve the proper translation of writings in a language at a specific point in time... The things suggested above are only the proverbial "tip of the iceberg" with respect to accomplishing this historical preservation -- it's a much larger project than it would seem... ...But it would be a very important project for Future Historians...

gwervc 3y ago
Some people are doing exactly this. The jibiki.fr project for example converted the Japanese-French dictionary from Cesselin into a digital form. OCR being what it is, plus change in orthography since the dictionary was published make human revisions necessary. It's a slow work as few people is interested in lexicography, moreso when dealing with "rare" languages or language pairs. There's a trove of old dictionaries in the public domain, for languages that sometimes have no computer support except being encoded in Unicode (I've in mind a dictionary written in the Amazigh alphabet). I've personally worked on the topic, on a niche language pair and writing system, yet it's far from being finished. With funding thing could go exponentially faster. This is something forgotten in the current AI talks (except from the artists point of view): the AI vendors would glob open data in an instant to improve their model but they don't contribute a dime to making the data available in the first place. And it's not hypable enough to raise money. Heck, with half a million of the Microsoft infinite money is throwing at OpenAI I could make your post a reality with a bunch of dictionaries I've in the top of my mind.
helpfulContrib 3y ago
[dead]

gwervc 3y ago

Some people are doing exactly this. The jibiki.fr project for example converted the Japanese-French dictionary from Cesselin into a digital form. OCR being what it is, plus change in orthography since the dictionary was published make human revisions necessary. It's a slow work as few people is interested in lexicography, moreso when dealing with "rare" languages or language pairs. There's a trove of old dictionaries in the public domain, for languages that sometimes have no computer support except being encoded in Unicode (I've in mind a dictionary written in the Amazigh alphabet). I've personally worked on the topic, on a niche language pair and writing system, yet it's far from being finished. With funding thing could go exponentially faster. This is something forgotten in the current AI talks (except from the artists point of view): the AI vendors would glob open data in an instant to improve their model but they don't contribute a dime to making the data available in the first place. And it's not hypable enough to raise money. Heck, with half a million of the Microsoft infinite money is throwing at OpenAI I could make your post a reality with a bunch of dictionaries I've in the top of my mind.

helpfulContrib 3y ago

[dead]