9 ms·
Indigenous engineers are using AI to preserve their culture
- deadbabe 2y agoFinally, a good use case for LLMs that isn’t just trying to anthropomorphize some already solved automation problem.
- DrillShopper 2y agoYeah but anthropomorphizing the LLM makes VCs spend stupid amounts of money!
- throwaway970598 2y agoIt seems like this could be incredibly fraught with danger once LLMs are involved (the story isn't exactly clear whether they are). If there are no surviving native speakers of a language (or very few) doing something like training an LLM to generate text in that language would run the risk of e.g. transferring English grammar to the vocabulary, and causing a hybrid language to become the dominant form of that language, because the LLM is used widely for study and the native-speaking elders are not.
- flocciput 2y agoIs that so bad? If the only way to ensure a language's preservation and continued use is to mangle and evolve it? I think that's kind of a neat concept, to be honest. To live is to change.
- tho23i423434 2y agoI wonder how useful this really is. No doubt, it's excellent for archiving, but that's not the same as "preserving" culture. If it's not alive and kicking it's not a culture IMO. You see this happen even with texts : once things start being written down, the actual knowledge tends to get lost (see India for example). This "AI to help low-resource languages" thing is a big deal in India too, but it just feels like another "jumla" for academics/techbros to make money. I mean, India has brutal/vicious policies that are out to destroy any and every language that's not English (since it's automatically a threat to central rule from Delhi), but pretty much no intellectual, either in India or the US, actually cares about the mass-wiping out of Indian languages by English... Not even the ones who go "ree bad British man destroyed India" on twitter all day.
- aussieguy1234 2y agoOnce an LLM knows an indigenous language, even if the last speaker dies out, future generations will be able to learn the language and use the LLM to converse with them in that language. Learning a new language is a good use case for LLMs, not just indigenous languages, but any language. As for your comment "ree bad British man destroyed India" this sound more like politics than anything of substance.
- bloak 2y agoYes, but the LLM is roughly equivalent to lossy compression of the corpus it is trained on so why wouldn't you preserve that actual corpus so that it can be used to train some better LLM, or something better than an LLM, in the future? (There may be a good answer to that question: perhaps, for example, the corpus can't be preserved for data protection reasons but the LLM trained on it can be preserved? For various reasons that doesn't seem very plausible, however.)
- crackalamoo 2y agoYou can do both. Preserving the corpus and building the LLM probably gives the best chance for future generations.
- BurningFrog 2y agoIt's really documenting the culture. A great thing in itself. I think of cultures and languages as tools. When they don't serve a purpose anymore, they should be replaced by something more functional.
- encipriano 2y agoThis demonstrates English utilitarianism basically won the philosophical war in Europe.
- meigwilym 2y agoWhat is the purpose of culture? It's a way of life. Arguably no culture has purpose, so do we force people to live in a different way? Cultures and languages die out because they're slowly (or revolutionarily quickly!) replaced by another. It's not like there are people out there speaking no language because their mother tongue has died out. And who gets to decide that a language has no purpose? I don't like the way this argument goes.
- joshdavham 2y agoThis is a fantastic use case for LLM’s. Also, godspeed to these researchers! There’s unfortunately not a lot of time left for many of these languages.
- userbinator 2y agos/preserve/hallucinate/ The next few decades are going to be really, really weird.
- sealeck 2y ago[flagged]
- lolinder 2y agoThis take us overly cynical in this case—the original and primary use case for LLMs is to model languages in a comprehensive way. It's literally in the name. Hallucinations rarely make up invalid grammar or invent non-existent words, what we're concerned about is facts, which isn't relevant at all when the goal is language preservation.
- userbinator 2y agoDepending on how powerful the language modeling is, I suspect it could lead to an LLM which will confidently and convincingly tell you how to say things like "floppy disk" and "SSD" in every extinct language and even those that went extinct long before computers ever existed, which is... interesting, but not exactly truth. I've seen LLMs hallucinate nonexistent things in programming languages. It's hard to believe it won't do the same to human ones.
- idunnoman1222 2y ago…Hallucinates a non-existent setting that could easily be added in a PR and merged tomorrow
- lolinder 2y agoImportantly, the hallucinating non-existent things in programming languages is still stringing together valid English words to make something that looks like it ought to be a correct concept. It doesn't construct new words from scratch. If a language model were asked what the word for "floppy disk" was in an extinct language and it invented a decent circumlocution, I don't think that would be a bad thing. People who are just engaging with the model as a way of connecting with their cultural heritage won't mind much if there is some creative application, and scholars are going to be aware of the limitations. Again, the misapplication of language models as databases is why hallucinations are a problem. This use case isn't treating the model as a database, it's treating the model as a model, so the consequences for hallucination are much smaller to the point of irrelevance.
- AnotherGoodName 2y agoFwiw this is the original usage of llms. The whole context awareness came about for the purpose of translations and the famous ‘attention is all you need’ paper was from the google translation team. Say what you will about llms being overhyped but this is the original core use case.
- highcountess 2y ago[dead]
- antics 2y agoI am one of these people! I am one of a handful of people who speak my ancestral language, Kiksht. I am lucky to be uniquely well-suited to this work, as I am (as far as I know) the lone person from my tribe whose academic research background is in linguistics, NLP, and ML. (We have, e.g., linguists, but very few computational linguists.) So far I have not had that much luck getting the models to learn the Kiksht grammar and morphology via in-context learning, I think the model will have to be trained on the corpus to actually work for it. I think this mostly makes sense, since they have functionally nothing in common with western languages. To illustrate the point a bit: the bulk of training data is still English, and in English, the semantics of a sentence are mainly derived from the specific order in which the words appear, mostly because it lost its cases some centuries ago. Its morphology is mainly "derivational" and mainly suffixal, meaning that words can be arbitrarily complicated by adding suffixes to them. So baked into English is word order that sometimes we insert words into sentences simply to make the word order sensible. e.g., when we say "it's raining outside", the "it's" refers to nothing at all—it is there entirely because the word order of English demands that it exists. Kiksht in contrast is completely different. Its semantics are nearly entirely derived from triple-prefixal structure of (in particular) verbs. Word ordering almost does not matter. There are, like, 12 tenses, and some of them require both a prefix and a reflective suffix. Verbs are often 1 or 2 characters, and with the prefix structure, a single verb can often be a complete sentence. And so on. I will continue working on this because I think it will eventually be of help. But right now the deep learning that has been most helpful to me has been to do things like computational typology. For example, discovering the "vowel inventory" of a language is shockingly hard. Languages have somewhat consistent consonants, but discovering all the varieties of `a` that one can say in a language is very hard, and deep learning is strangely good at it.
- koolba 2y agoAccording to Wikipedia there were 69 fluent speakers of Kiksht in 1990, and the last one passed away in 2012. How did you learn the language? https://en.wikipedia.org/wiki/Upper_Chinook_language https://en.wikipedia.org/wiki/Upper_Chinook_language
- 2y ago
- Mengkudulangsat 2y agoOn an side note, can anyone recommend an AI tool I can use to learn a random niche language as a hobby (e.g. Toki Pona)?
- diggan 2y agoI think what most LLMs have in common, is that they're pretty good at translations out-of-the-box, even the general purpose ones. Probably even the free versions of ChatGPT, Claude or fully free DeepSeek can help you out pretty well with that.
- thomasfromcdnjs 2y agoI'm an Australian indigenous and have been slowly working on this problem in my own way for a few years. https://github.com/australia/mobtranslate.com/ https://github.com/australia/mobtranslate.com/ In it's current iteration the homepage is just running dictionaries through OpenAI. (my tribes dictionary fits in a 100k context window) My old ambitions can be found somewhat here -> https://github.com/australia/mobtranslate-server https://github.com/australia/mobtranslate-server That being said, the OpenAI models do a fantastic job at translating sentences so I've put my own model research further to the back. (will try to find some examples) I can't speak of the true preservation, not many native speakers left, but in my mind that's not even all that important from a personal/cultural perspective. If the youth who are interested in learning more about their language have a nice interface with 70-80% accurate results and they enjoy doing/learning it then that is a win to me. (and kind of how language evolves anyway) (the noun replacement seems to work great, but grammar is obviously wishy-washy) (At this point, I just rushed to get my tribes dictionary crawlable so hopefully it will be in a few models next training phases)
- prawn 2y agoI was looking to name a property recently and tried ChatGPT for suggestions on a theme of geographical features in various languages. I had it try Kaurna (Adelaide area) on a whim and was pleasantly surprised (given that Google Translate doesn't cover it, I guess) to find that it gave loads of relevant suggestions without any issue. Any that I picked to selectively check verified with Kaurna dictionaries OK.
- iamnotsure 2y agome too
- romaaeterna 2y ago"His dream is to revive dying languages..." The content of modern culture is too much for dying or ancient languages, and what you actually get is English/modern thoughtspace expressed in the lexicon of an until-now separate culture. This flood of spam destroys what was unique and interesting about the culture, and "skin-suits" it.
- rexpop 2y agoMaybe, unless some form of Sapir-Whorf is true, and the lexica—and grammar, and phonology, etc.—impose an inalielable umwelt, one inaccessible to hegemony.
- romaaeterna 2y agoThey certainly do. But languages are also very elastic. If you speak modern German, for example, you'll find that it's very similar to modern American English in thought and expression. But this was not true of late 19th-century German and 19th-century English. There has unfortunately been substantial thought convergence due to mass communication. If all you speak is English, you may also find that the current dialectical forms are all much weaker and samey compared to the older forms. Anyway, in resurrected ancient languages, this modern samey-ness is a problem. Go read Latin Wikipedia, for example. It's much more like reading modern English authors (though with Latin words and grammar) than it is like reading classical authors. -- I add a natural language translation (from ChatGPT) of your statement for anyone who doesn't share your lexica: > "Maybe, unless some version of the Sapir-Whorf hypothesis is correct, and language—along with its grammar, sounds, and other aspects—creates an inescapable worldview that can't be dominated."
- rexpop 2y agoChatGPT has misinterpreted me. By "inalienable unwelt," I mean it in the sense of "inalienable rights" that cannot be removed by external circumstance. By "inaccessible to hegemony," I mean an umwelt which cannot be perceived by speakers of the dominant language.
- temptemptemp111 2y ago[dead]
- reportgunner 2y agoI don't understand, how can they use AI to preserve their culture when AI was never a part of it ?
- lolinder 2y agoThe same way linguists are already using audio recordings to preserve dying languages that belong to people who have never had access to audio recordings. You can make a record of a culture in a medium that they didn't/don't have themselves.
- krapp 2y agoAI fabricates and hallucinates (or whatever term you want to use) by design - it doesn't provide a neutral record the way audio recordings might. I think a better question might be what, specifically, does AI bring to the table that justifies its inherent inaccuracy? Why not just use audio recordings, etc?
- lolinder 2y agoLLMs, when trained on a language, provide a neutral representation of the statistics of that data set. Used correctly, that makes them perfect for this kind of task. You're reacting to global misuse of llms as databases, but llms as language models is what they are designed for. It's in the name. Used judiciously as a model, there's no problem. They only become a problem when people try to treat them as a database.
- reportgunner 2y agoI have never heard of that, can you elaborate ?
- lolinder 2y agoI mean, it's what it says on the tin: linguists go into remote areas and take recordings of dying languages to preserve them. They'll sometimes prompt for specific words and sentences, other times just record people speaking naturally. Then they'll also write down the rules of the grammar and the vocabulary (another example of using a medium they may not themselves have).
- sdsd 2y agoI've always resented the way "traditional" cultures are "preserved". It's like when people want to protect "authentic" locales from the corrupting influence of tourism. I'm grateful that my own culture is not (yet) seen primarily through this lens. Imagine the day when legislation makes exceptions for fentanyl as a traditional ritual substance of nomadic Trailer Americans. To me the beauty of these things is in their liveliness, in the aspiration to flourish and grow, not merely to conserve a little longer, to spend one more night with terminal cancer before the inevitable.
- tomp 2y agoTerrible title. Every engineer is indigenous somewhere.
- rexpop 2y agoIndigeneity generally refers to the descendants of people who inhabited a territory prior to colonization or the formation of a country, often those who are disadvantaged as a result, and who continue to inhabit the land. Connection to the land is a fundamental element of indigeneity, as is the specific condition of dispossession by colonization: indigeneity, as it is understood today, emerges in relation to colonial processes. So, although all distinct ethnicities may originate from specific places and times, indigeneity as a political and social identity is meaningful only in the context of colonial domination and resistance. Not every engineer is engaged in an ongoing struggle for sovereignty against colonizing powers.