11 ms·

Using AI to decipher languages that remain a mystery to us

83 points by Hooke 5y ago

Bayart 5y ago
It reminds me of the first great decipherment, that of hieroglyphic Egyptian. Young's quantitative approach gave a few insights that proved useful later on, but that approach was ultimately limited and wasn't leading anywhere. It wasn't until Champollion brought Coptic to the table that it was unlocked.
- Psyladine 5y ago
  I was going to comment on the Coptic connection; Hieroglyphs would have remained a mystery due to their basically vowel-less structure; it was an interpreted language[0]. I wonder to what extent this AI could crack that connection, since we have a more or less complete knowledge of what it ought to produce. [0]https://www.youtube.com/watch?v=J-K5OjAkiEA https://www.youtube.com/watch?v=J-K5OjAkiEA
humility 5y ago
Oh my, I've wondered about this a lot since I come from a Sanskrit background and I love etymology. I'm also cataloguing the superb similarities between Sanskrit and English despite several mutations spanning millennia. The optimist in me thinks this is the written script for an early version of Proto-Indo-European. Would love to see what like minded people think :)
- robinsoh 5y ago
  > I come from a Sanskrit background Meaning a scholar of Sanskrit or something else? > The optimist in me thinks this is the written script for an early version of Proto-Indo-European Optimist in what sense?
  - humility 5y ago
    > meaning a scholar of Sanskrit or something else? I studied Sanskrit for 5 years in middle school, so I'm familiar with more than the basics. > Optimist in what sense The one who wants to see a script for PIE discovered :)
    - robinsoh 5y ago
      > The one who wants to see a script for PIE discovered :) I didn't quite understand. Is this related to what the article says: " The discovery of a civilization of people who lived before the Vedic people upended the story of India. Given that it undermines their claims of indigeneity, proponents of Hindutva — the most mainstream strain of Hindu nationalism — balk at the theory of a pre-Vedic civilization, even as evidence for it accumulates across disciplines, including archaeology, genetics, and linguistics. " Is Sanskrit and Hindutva and PIE related?
      - humility 5y ago
        I don't know how Hindutva is related to this discussion. I'm just interested in this from a coherence point of view. I believe that learning a super language like PIE would help me understand a lot about human relation to language and vice versa.
        robinsoh 5y ago
        The article mentioned Hindutva as I pasted. If I understand correctly, Hindutva people have a vested interest to want this script to be Sanskrit related as opposed to an indigenous Indian language. Is that correct?
        humility 5y ago
        > If I understand correctly, Hindutva people have a vested interest to want this .... Don't know, don't care about this 40 year old concept as much as I do for a 5000 year proto language.
      - mcswell 5y ago
- arnsholt 5y ago
  Anything IE is extremely unlikely as a language for the Indus seals; it only really fits with an out of India model of Indo-European, which is outright rejected by all serious scholars AFAIK. If we're going to look for contemporary relatives of the Indus language, the serious candidate is Dravidian. The Dravidian family was in India around the right time, and was definitely spoken in a larger area prior to the Aryan invasions.
  - hasmanean 5y ago
    I don’t think the Indus Valley seals are a script at all. They might be more like corporate logos. Try deciphering the english language from a bunch of billboards and pop/cans. Even if you figure out the alphabet there aren’t enough words to construct a single sentence. All the AI in the universe won’t help when the problem is lack of real data.
    - arnsholt 5y ago
      Well, names and titles are still language, but I agree with your actual point: the Indus inscriptions seem to be too short and probably not general enough to allow an in-depth understanding without finding new, longer texts.
    - mcswell 5y ago
      That was Sproat et al's point. (Sorry, I can't remember the other authors, I just happen to know Richard Sproat.)
fooker 5y ago
How can 'AI' possibly crack the code without training data?
- tapland 5y ago
  Using other languages as a basis.
  - EdwardDiego 5y ago
    Which languages though? There's a lot of languages in and around Punjab.
- ffhhj 5y ago
  By finding a similar language.
  - EdwardDiego 5y ago
    Which language though? There's so many. It's already been tied to Proto-Dravidian, or to Tamil, or it could be extinct.
    - thechao 5y ago
      One way to think of this: have the computer learn how to translate between any two languages, i.e., learn how to "build translators", and then attempt to translate between the unknown language and every other language.
  - usrusr 5y ago
    How would you identify similarity though when you know nothing about the encoding?
    - eru 5y ago
      In general, you could try many encodings roughly in order of algorithmic complexity. Not sure if that's any use here. The really big problem is lack of data. With sufficient amounts of data and the assumption that we are dealing with human writing, you can probably figure out the encoding.
swara11 5y ago
rvense 5y ago
This article is horrible. We have no idea what's on those seals. If it's even a script, it might be personal names. The inscriptions are all very short. We have no idea how the script is structured, if it's syllabic or what. We have absolutely no idea what language it's written in. There is nothing to latch on to, no way to anchor the findings in the real world. The problem with applying AI to this is that it will give an answer, but there's absolutely no way to verify it's a reasonable one.
- scim-knox-twox 5y ago
  I was expected this, even title is a clickbait.
- melony 5y ago
  We can break codes by cryptography, a network pre trained on all human languages should be able to derive insights unsupervised.
  - rnhmjoj 5y ago
    This is akin to breaking a one-time-pad cipher: you could easily come up with many possible plaintext but there is no way of verifying one is correct.
  - tgv 5y ago
    If you can build a network, that can crack Enigma after being trained on German and preceding encryption methods, then you might have made a step towards this. But note that cracking enigma "only" took a few people a few months, that these scripts have eluded many people for a much longer time, and that the source language of Enigma was known, as was its script.
    - eru 5y ago
      I get your general point, but the Enigma (or any encryption) is a pretty bad example. Encrypted text 'wants' to be obscure, even to those who know the system but lack the key. Most writing systems 'want' to be decipherable to people in the know. In addition, Enigma has a fairly short key length by modern standards, and lots of accidental flaws that make it even easier to crack. (Eg an automated system that knows a lot of German _might_ be able to figure out that the enigma never encrypts a letter to itself, even without seeing any schematics.)
EdwardDiego 5y ago
Cool, it can decipher Linear B, like humans, because there's a bunch of data points for Linear B (IIRC around 6.5K inscriptions in the corpus) and because it's known that it's written in Mycenaean Greek, and Ancient Greek gives us a place to start working backwards from. But then there's Linear A. Only has 1500 inscriptions or so. And no translation. So, Harappan/Indus script has about 4K known inscriptions. And descendant languages to work backwards from are still unknown. Well, I wish them good luck. But I don't know if ML can do any better than humans. It might be able to do what humans can faster, but I doubt it can do what we can't. That said, I would love to be proven wrong, because I love the idea of the Indus valley civilisation being where "all the wise men in boats" (to paraphrase Pratchett) came from, given the obvious proximity of the Indus Valley to the Persian Gulf and thus Mesopotamia. Bonus points if someone translates a Harappan inscription and it says "By the way, if some guy called Noah turns up in a boat, claiming to have been spared from a global flood, he's a bit nutty, it was just a heavy monsoon."
- eru 5y ago
  > Well, I wish them good luck. But I don't know if ML can do any better than humans. It might be able to do what humans can faster, but I doubt it can do what we can't. Being faster might be enough? Who knows, perhaps some human spending a few centuries in deep contemplation of Linear A might crack it?
  - yorwba 5y ago
    Humans have already spent combined person-centuries trying to crack Linear A, and been able to guess a few words from context. If you use the sound values from Linear B (which can be guessed via Greek), you can even try to pronounce them out loud. Unfortunately, that doesn't help at all unless you can find a related language to help guess more words. If it's a language isolate without living relatives, that would be simply impossible. However, that doesn't mean that trying to solve it with ML is completely useless. If you can do a century's worth of hypothesis-testing in a fraction of the time to discover that none of them work, at least humans will waste less time trying to solve an unsolvable problem.
  - 5y ago
PeterisP 5y ago
"The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data." -- John Tukey
gullevek 5y ago
If the headline can be answered with No. It is No.
- taneq 5y ago
  You should write an article about it. Maybe call it "Is Betteridge's Law of Headlines really true?"
dotancohen 5y ago
I believe that Hoshi Sato developed models for real-time translation of spoken language using very limited data samples without translations. It might be an application of the research mentioned here. The timeframes fit.
- mcswell 5y ago
  The timeframe being a few hundred years in our future?
inawarminister 5y ago
Huh. The finding that Iberian is not a close language to Basque is surprising, but then again the Romans did not put the two of them as one nation- like they did with Aquitanians (Southwest France today) and Basque territory hmm. Pilus, pirus, pilis. This is intriguing. But one word doesn't signify the membership of a language in another language family. Is it possible that Tamil itself adopted the IVC term for elephants? Of course, since in all likelihood proto-Tamil speakers absorbed many refugees from IVC, there should be many terms shared between the two languages, even if they're from different language families Still interesting to speculate.
deleted 5y ago
[deleted]
sam_goody 5y ago
Betteridge's law of headlines: "Any headline that ends in a question mark can be answered by the word no." As a kid who could read Phoenician script (similar to ancient Hebrew), I tried to decipher the Ashmanezer tablet, which is written in a Semitic language and has many proper nouns. Despite that, there were different ways to interpret the inscription. Taught me just how amazing and difficult language is. (In the Wikipedia page[1], there is one translation given as though it is empirical. It's not.) [1]: https://en.wikipedia.org/wiki/Eshmunazar_II https://en.wikipedia.org/wiki/Eshmunazar_II
tpoacher 5y ago
This article is 3566 words. 15 pages long when copied onto libreoffice. I like reading as much as the next person, but this kind of articles is starting to get ridiculous. It's like, it starts with an interesting but otherwise straightforward title, you click, and then you're presented with a novel. One structured in such a way so as to make trying to glance to find the relevant sections yields zero information. Nope, you are required to read all about how someone grew without a dog and how their grandmother had lovely spots on her hands and her skin was like paper, before you can even get an inkling where the information lies in the article, if it's there at all. I've read countless articles like this by now, where you go through the whole 'novel', and the information you were looking for is on paragraph 67 out of 103, and it's basically the line "Well, we don't really know." (ノಠ益ಠ)ノ彡┻━┻ Here, let me tell you about the story when I once added 1 + 1 together. You'll never believe the result: "Jonathan was a keen hiker. When his wife got pregnant in the mid nineties, he could never imagine that his great-great-great-great-great grandson would ever be faced with the 1+1 problem [...]"
- eganist 5y ago
  > I've read countless articles like this by now, where you go through the whole 'novel', and the information you were looking for is on paragraph 67 out of 103, and it's basically the line "Well, we don't really know." That's basically the outcome here. It talks briefly about a few different approaches and doesn't really touch on the likelihood of success. I just grepped for " AI" and read the few paragraphs that popped up. Although looking for "algorithm" proved slightly more interesting: > For now, the mysteries of the Indus script continue to elude decipherment. Last year, in a follow-up paper to their work automating the decoding of Ugaritic and Linear B, Luo and his team made a small but crucial advance: an algorithm aimed at identifying possible related languages of undeciphered writing systems. Potentially, this could help address the problem of deciphering scripts that don’t yet have a known language they can be compared against. When Luo and his team tested their model on the Iberian language, which has historically been linked to Basque, their findings suggested the two languages were not in fact close enough to be related — a conclusion that corroborated recent scholarship on the matter. I'd want to see this applied to dolphin clicks. Not sure it would get anywhere, but if an existing human language turns out to have similar grammatical and linguistic constructs, it might get the ball rolling on communicating.
thomasfromcdnjs 5y ago
I'm currently working on translating my tribes indigenous language (Australia) -> https://github.com/australia/mobtranslate-server https://github.com/australia/mobtranslate-server When I'm finished, I wanted to play around with some ML exercises to see if I could derive anything useful in understanding it. Besides this article does anyone know of any practical/simple attempts at running ML over historical languages?
- mcswell 5y ago
  ML over historical languages is no different from ML over currently spoken (and written) languages, with one crucial exception: if the language is still in use, you can get a bigger corpus. About dictionary making: there are several programs "out there" that are already set up for making dictionaries, probably better than going from scratch with YAML. SIL's FLEx is very good; their older Shoebox program is also useable, but like the C programming language, you can shoot yourself in the foot. There's another one out of East Africa, but I can't remember its name. I don't know much about Australian languages, but I think most of them have a lot of morphology. If yours does, then you'll want to do something about inflected forms of words (think English walk, walks, walked, walking); you probably don't want to store all of those forms in your dictionary, instead you'll want to choose a base form from which the others can be derived. Again, tools like FLEx allow you to create morphological parsers that help with that. Finally, I'd suggest that if you haven't already done so, you should take some courses in linguistics. It will make it much easier to reason about your lexicon, the language's grammar (syntax and morphology), and so forth.
pessimizer 5y ago
Has anybody come up with AI tactics specifically aimed at simply generating new grammatical sentences from a single-language corpus? Meaning that the results aren't judged by meaning or coherence, but by adherence to a grammar, with an attempt to maximize variety? The goal would be to do that well with languages where we know the grammar and can evaluate the results, and through that find a general tactic. Maybe find a few general theories on how much input that takes, and the character of that input. If you can extract the grammatical rules with reasonable certainty (not likely with a collection of seals, of course) figuring out the content would just be Mad Libs.
- zozbot234 5y ago
  > Has anybody come up with AI tactics specifically aimed at simply generating new grammatical sentences from a single-language corpus? Meaning that the results aren't judged by meaning or coherence, but by adherence to a grammar, with an attempt to maximize variety? Just training an LSTM deep network on the corpus is enough to do that. GPT-3 is much the same, with a lot more training data and a less computationally intensive network architecture than LSTM, to make that huge amount of training data useful.
  - phreeza 5y ago
    A language model approach will learn grammatical/syntactic and semantic patterns at the same time. So "dog walks man" will be less likely than "man walks dog", even though both are valid, grammatically. I think what op was asking for was a mechanism to purely learn grammar, which is more in the realm of GOFAI/old school NLP.
- phreeza 5y ago
  I think "grammar induction" is a good search term to get started here.

83 points by Hooke 5y ago

It reminds me of the first great decipherment, that of hieroglyphic Egyptian. Young's quantitative approach gave a few insights that proved useful later on, but that approach was ultimately limited and wasn't leading anywhere. It wasn't until Champollion brought Coptic to the table that it was unlocked.

Psyladine 5y ago
I was going to comment on the Coptic connection; Hieroglyphs would have remained a mystery due to their basically vowel-less structure; it was an interpreted language[0]. I wonder to what extent this AI could crack that connection, since we have a more or less complete knowledge of what it ought to produce. [0]https://www.youtube.com/watch?v=J-K5OjAkiEA https://www.youtube.com/watch?v=J-K5OjAkiEA

humility 5y ago

Oh my, I've wondered about this a lot since I come from a Sanskrit background and I love etymology. I'm also cataloguing the superb similarities between Sanskrit and English despite several mutations spanning millennia. The optimist in me thinks this is the written script for an early version of Proto-Indo-European. Would love to see what like minded people think :)

robinsoh 5y ago
> I come from a Sanskrit background Meaning a scholar of Sanskrit or something else? > The optimist in me thinks this is the written script for an early version of Proto-Indo-European Optimist in what sense?
- humility 5y ago
  > meaning a scholar of Sanskrit or something else? I studied Sanskrit for 5 years in middle school, so I'm familiar with more than the basics. > Optimist in what sense The one who wants to see a script for PIE discovered :)
  - robinsoh 5y ago
    > The one who wants to see a script for PIE discovered :) I didn't quite understand. Is this related to what the article says: " The discovery of a civilization of people who lived before the Vedic people upended the story of India. Given that it undermines their claims of indigeneity, proponents of Hindutva — the most mainstream strain of Hindu nationalism — balk at the theory of a pre-Vedic civilization, even as evidence for it accumulates across disciplines, including archaeology, genetics, and linguistics. " Is Sanskrit and Hindutva and PIE related?
    - humility 5y ago
      I don't know how Hindutva is related to this discussion. I'm just interested in this from a coherence point of view. I believe that learning a super language like PIE would help me understand a lot about human relation to language and vice versa.
      - robinsoh 5y ago
        The article mentioned Hindutva as I pasted. If I understand correctly, Hindutva people have a vested interest to want this script to be Sanskrit related as opposed to an indigenous Indian language. Is that correct?
        humility 5y ago
        > If I understand correctly, Hindutva people have a vested interest to want this .... Don't know, don't care about this 40 year old concept as much as I do for a 5000 year proto language.
    - mcswell 5y ago
arnsholt 5y ago
Anything IE is extremely unlikely as a language for the Indus seals; it only really fits with an out of India model of Indo-European, which is outright rejected by all serious scholars AFAIK. If we're going to look for contemporary relatives of the Indus language, the serious candidate is Dravidian. The Dravidian family was in India around the right time, and was definitely spoken in a larger area prior to the Aryan invasions.
- hasmanean 5y ago
  I don’t think the Indus Valley seals are a script at all. They might be more like corporate logos. Try deciphering the english language from a bunch of billboards and pop/cans. Even if you figure out the alphabet there aren’t enough words to construct a single sentence. All the AI in the universe won’t help when the problem is lack of real data.
  - arnsholt 5y ago
    Well, names and titles are still language, but I agree with your actual point: the Indus inscriptions seem to be too short and probably not general enough to allow an in-depth understanding without finding new, longer texts.
  - mcswell 5y ago
    That was Sproat et al's point. (Sorry, I can't remember the other authors, I just happen to know Richard Sproat.)

fooker 5y ago

How can 'AI' possibly crack the code without training data?

tapland 5y ago
Using other languages as a basis.
- EdwardDiego 5y ago
  Which languages though? There's a lot of languages in and around Punjab.
ffhhj 5y ago
By finding a similar language.
- EdwardDiego 5y ago
  Which language though? There's so many. It's already been tied to Proto-Dravidian, or to Tamil, or it could be extinct.
  - thechao 5y ago
    One way to think of this: have the computer learn how to translate between any two languages, i.e., learn how to "build translators", and then attempt to translate between the unknown language and every other language.
- usrusr 5y ago
  How would you identify similarity though when you know nothing about the encoding?
  - eru 5y ago
    In general, you could try many encodings roughly in order of algorithmic complexity. Not sure if that's any use here. The really big problem is lack of data. With sufficient amounts of data and the assumption that we are dealing with human writing, you can probably figure out the encoding.

swara11 5y ago

rvense 5y ago

This article is horrible. We have no idea what's on those seals. If it's even a script, it might be personal names. The inscriptions are all very short. We have no idea how the script is structured, if it's syllabic or what. We have absolutely no idea what language it's written in. There is nothing to latch on to, no way to anchor the findings in the real world. The problem with applying AI to this is that it will give an answer, but there's absolutely no way to verify it's a reasonable one.

scim-knox-twox 5y ago
I was expected this, even title is a clickbait.
melony 5y ago
We can break codes by cryptography, a network pre trained on all human languages should be able to derive insights unsupervised.
- rnhmjoj 5y ago
  This is akin to breaking a one-time-pad cipher: you could easily come up with many possible plaintext but there is no way of verifying one is correct.
- tgv 5y ago
  If you can build a network, that can crack Enigma after being trained on German and preceding encryption methods, then you might have made a step towards this. But note that cracking enigma "only" took a few people a few months, that these scripts have eluded many people for a much longer time, and that the source language of Enigma was known, as was its script.
  - eru 5y ago
    I get your general point, but the Enigma (or any encryption) is a pretty bad example. Encrypted text 'wants' to be obscure, even to those who know the system but lack the key. Most writing systems 'want' to be decipherable to people in the know. In addition, Enigma has a fairly short key length by modern standards, and lots of accidental flaws that make it even easier to crack. (Eg an automated system that knows a lot of German _might_ be able to figure out that the enigma never encrypts a letter to itself, even without seeing any schematics.)

EdwardDiego 5y ago

Cool, it can decipher Linear B, like humans, because there's a bunch of data points for Linear B (IIRC around 6.5K inscriptions in the corpus) and because it's known that it's written in Mycenaean Greek, and Ancient Greek gives us a place to start working backwards from. But then there's Linear A. Only has 1500 inscriptions or so. And no translation. So, Harappan/Indus script has about 4K known inscriptions. And descendant languages to work backwards from are still unknown. Well, I wish them good luck. But I don't know if ML can do any better than humans. It might be able to do what humans can faster, but I doubt it can do what we can't. That said, I would love to be proven wrong, because I love the idea of the Indus valley civilisation being where "all the wise men in boats" (to paraphrase Pratchett) came from, given the obvious proximity of the Indus Valley to the Persian Gulf and thus Mesopotamia. Bonus points if someone translates a Harappan inscription and it says "By the way, if some guy called Noah turns up in a boat, claiming to have been spared from a global flood, he's a bit nutty, it was just a heavy monsoon."

eru 5y ago
> Well, I wish them good luck. But I don't know if ML can do any better than humans. It might be able to do what humans can faster, but I doubt it can do what we can't. Being faster might be enough? Who knows, perhaps some human spending a few centuries in deep contemplation of Linear A might crack it?
- yorwba 5y ago
  Humans have already spent combined person-centuries trying to crack Linear A, and been able to guess a few words from context. If you use the sound values from Linear B (which can be guessed via Greek), you can even try to pronounce them out loud. Unfortunately, that doesn't help at all unless you can find a related language to help guess more words. If it's a language isolate without living relatives, that would be simply impossible. However, that doesn't mean that trying to solve it with ML is completely useless. If you can do a century's worth of hypothesis-testing in a fraction of the time to discover that none of them work, at least humans will waste less time trying to solve an unsolvable problem.
- 5y ago

PeterisP 5y ago

"The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data." -- John Tukey

gullevek 5y ago

If the headline can be answered with No. It is No.

taneq 5y ago
You should write an article about it. Maybe call it "Is Betteridge's Law of Headlines really true?"

dotancohen 5y ago

I believe that Hoshi Sato developed models for real-time translation of spoken language using very limited data samples without translations. It might be an application of the research mentioned here. The timeframes fit.

mcswell 5y ago
The timeframe being a few hundred years in our future?

inawarminister 5y ago

Huh. The finding that Iberian is not a close language to Basque is surprising, but then again the Romans did not put the two of them as one nation- like they did with Aquitanians (Southwest France today) and Basque territory hmm. Pilus, pirus, pilis. This is intriguing. But one word doesn't signify the membership of a language in another language family. Is it possible that Tamil itself adopted the IVC term for elephants? Of course, since in all likelihood proto-Tamil speakers absorbed many refugees from IVC, there should be many terms shared between the two languages, even if they're from different language families Still interesting to speculate.

deleted 5y ago

[deleted]

sam_goody 5y ago

Betteridge's law of headlines: "Any headline that ends in a question mark can be answered by the word no." As a kid who could read Phoenician script (similar to ancient Hebrew), I tried to decipher the Ashmanezer tablet, which is written in a Semitic language and has many proper nouns. Despite that, there were different ways to interpret the inscription. Taught me just how amazing and difficult language is. (In the Wikipedia page[1], there is one translation given as though it is empirical. It's not.) [1]: https://en.wikipedia.org/wiki/Eshmunazar_II https://en.wikipedia.org/wiki/Eshmunazar_II

tpoacher 5y ago

This article is 3566 words. 15 pages long when copied onto libreoffice. I like reading as much as the next person, but this kind of articles is starting to get ridiculous. It's like, it starts with an interesting but otherwise straightforward title, you click, and then you're presented with a novel. One structured in such a way so as to make trying to glance to find the relevant sections yields zero information. Nope, you are required to read all about how someone grew without a dog and how their grandmother had lovely spots on her hands and her skin was like paper, before you can even get an inkling where the information lies in the article, if it's there at all. I've read countless articles like this by now, where you go through the whole 'novel', and the information you were looking for is on paragraph 67 out of 103, and it's basically the line "Well, we don't really know." (ノಠ益ಠ)ノ彡┻━┻ Here, let me tell you about the story when I once added 1 + 1 together. You'll never believe the result: "Jonathan was a keen hiker. When his wife got pregnant in the mid nineties, he could never imagine that his great-great-great-great-great grandson would ever be faced with the 1+1 problem [...]"

eganist 5y ago
> I've read countless articles like this by now, where you go through the whole 'novel', and the information you were looking for is on paragraph 67 out of 103, and it's basically the line "Well, we don't really know." That's basically the outcome here. It talks briefly about a few different approaches and doesn't really touch on the likelihood of success. I just grepped for " AI" and read the few paragraphs that popped up. Although looking for "algorithm" proved slightly more interesting: > For now, the mysteries of the Indus script continue to elude decipherment. Last year, in a follow-up paper to their work automating the decoding of Ugaritic and Linear B, Luo and his team made a small but crucial advance: an algorithm aimed at identifying possible related languages of undeciphered writing systems. Potentially, this could help address the problem of deciphering scripts that don’t yet have a known language they can be compared against. When Luo and his team tested their model on the Iberian language, which has historically been linked to Basque, their findings suggested the two languages were not in fact close enough to be related — a conclusion that corroborated recent scholarship on the matter. I'd want to see this applied to dolphin clicks. Not sure it would get anywhere, but if an existing human language turns out to have similar grammatical and linguistic constructs, it might get the ball rolling on communicating.

thomasfromcdnjs 5y ago

I'm currently working on translating my tribes indigenous language (Australia) -> https://github.com/australia/mobtranslate-server https://github.com/australia/mobtranslate-server When I'm finished, I wanted to play around with some ML exercises to see if I could derive anything useful in understanding it. Besides this article does anyone know of any practical/simple attempts at running ML over historical languages?

mcswell 5y ago
ML over historical languages is no different from ML over currently spoken (and written) languages, with one crucial exception: if the language is still in use, you can get a bigger corpus. About dictionary making: there are several programs "out there" that are already set up for making dictionaries, probably better than going from scratch with YAML. SIL's FLEx is very good; their older Shoebox program is also useable, but like the C programming language, you can shoot yourself in the foot. There's another one out of East Africa, but I can't remember its name. I don't know much about Australian languages, but I think most of them have a lot of morphology. If yours does, then you'll want to do something about inflected forms of words (think English walk, walks, walked, walking); you probably don't want to store all of those forms in your dictionary, instead you'll want to choose a base form from which the others can be derived. Again, tools like FLEx allow you to create morphological parsers that help with that. Finally, I'd suggest that if you haven't already done so, you should take some courses in linguistics. It will make it much easier to reason about your lexicon, the language's grammar (syntax and morphology), and so forth.

pessimizer 5y ago

Has anybody come up with AI tactics specifically aimed at simply generating new grammatical sentences from a single-language corpus? Meaning that the results aren't judged by meaning or coherence, but by adherence to a grammar, with an attempt to maximize variety? The goal would be to do that well with languages where we know the grammar and can evaluate the results, and through that find a general tactic. Maybe find a few general theories on how much input that takes, and the character of that input. If you can extract the grammatical rules with reasonable certainty (not likely with a collection of seals, of course) figuring out the content would just be Mad Libs.

zozbot234 5y ago
> Has anybody come up with AI tactics specifically aimed at simply generating new grammatical sentences from a single-language corpus? Meaning that the results aren't judged by meaning or coherence, but by adherence to a grammar, with an attempt to maximize variety? Just training an LSTM deep network on the corpus is enough to do that. GPT-3 is much the same, with a lot more training data and a less computationally intensive network architecture than LSTM, to make that huge amount of training data useful.
- phreeza 5y ago
  A language model approach will learn grammatical/syntactic and semantic patterns at the same time. So "dog walks man" will be less likely than "man walks dog", even though both are valid, grammatically. I think what op was asking for was a mechanism to purely learn grammar, which is more in the realm of GOFAI/old school NLP.
phreeza 5y ago
I think "grammar induction" is a good search term to get started here.