4 ms·

Thank you so much for this course. I plan to go through it properly. I have a search problem of my own and I have had a hard time applying what I have learnt (

Thank you so much for this course. I plan to go through it properly. I have a search problem of my own and I have had a hard time applying what I have learnt (including the coursera DL specialization). The chief characteristics are: (a) It is a fuzzy search of a corpus that is in a non-English language. (b) The search should be able to run on a mobile phone _offline_. Is this possible? Can training be done elsewhere and transferred to TinyML or some such? What would be a good forum to go seeking answers?

devnonymous 4y ago
If the volume of data fits on a mobile phone for it to be offline, perhaps you don't need deep learning?
- sriram_malhar 4y ago
  Perhaps. What alternative would you suggest? The search terms are fuzzy, and there are too many variants (not exactly misspellings) for me to encode them explicitly. So I thought I'd rather learn from a crowd-sourced corpus. In my case, I don't want the model to be general. I can afford for it to be like a database index, tailored to that data.
leobg 4y ago
Have you tried... a) BM25 after some preprocessing (lemmatization etc.) b) fastText / GloVe (possibly weighted by BM25) The results can be surprisingly good. Often no need to bother with big language models or GPUs.
- sriram_malhar 4y ago
  As far as I understand, BM25 is not for fuzzy searches. For a bit more context, the search terms are in the English script, but the words are basically the closest-sounding transcriptions of sounds in various Indian languages. Different people may render the same word differently in an English transcription. But there's enough crowd-sourced data to account for the ways in which words can vary. For the same reason, GloVe is of not much use to me.
  - leobg 4y ago
    OK, I understand. That is a different problem indeed. Although I am not sure that BERT would be of much more help there either, unless you have quite a large training corpus. The simplest/cheapest approach might be some kind of transcription normalization. As you say, what is introducing ambiguity here is the act of transcribing these sounds into the English alphabet. There’s not a single source language/alphabet? Also, I am not sure if fastText/GloVe really would not pick up the semantic similarity in spite of differing transcriptions as long as you have a large enough training corpus. I’d experiment with the settings here (bag of words/skipgram, min/max lengths etc.).

devnonymous 4y ago

If the volume of data fits on a mobile phone for it to be offline, perhaps you don't need deep learning?

sriram_malhar 4y ago
Perhaps. What alternative would you suggest? The search terms are fuzzy, and there are too many variants (not exactly misspellings) for me to encode them explicitly. So I thought I'd rather learn from a crowd-sourced corpus. In my case, I don't want the model to be general. I can afford for it to be like a database index, tailored to that data.

leobg 4y ago

Have you tried... a) BM25 after some preprocessing (lemmatization etc.) b) fastText / GloVe (possibly weighted by BM25) The results can be surprisingly good. Often no need to bother with big language models or GPUs.

sriram_malhar 4y ago
As far as I understand, BM25 is not for fuzzy searches. For a bit more context, the search terms are in the English script, but the words are basically the closest-sounding transcriptions of sounds in various Indian languages. Different people may render the same word differently in an English transcription. But there's enough crowd-sourced data to account for the ways in which words can vary. For the same reason, GloVe is of not much use to me.
- leobg 4y ago
  OK, I understand. That is a different problem indeed. Although I am not sure that BERT would be of much more help there either, unless you have quite a large training corpus. The simplest/cheapest approach might be some kind of transcription normalization. As you say, what is introducing ambiguity here is the act of transcribing these sounds into the English alphabet. There’s not a single source language/alphabet? Also, I am not sure if fastText/GloVe really would not pick up the semantic similarity in spite of differing transcriptions as long as you have a large enough training corpus. I’d experiment with the settings here (bag of words/skipgram, min/max lengths etc.).