5 ms·

I build and play lyres, and there's a thriving community of lyre players in Japan who have put a lot of videos on YouTube. None of them show up in my searches

I build and play lyres, and there's a thriving community of lyre players in Japan who have put a lot of videos on YouTube. None of them show up in my searches for lyre music, which quickly devolve into people playing funk bass and flying homebuilt airplanes, both of which I like, but still! I had to use NewPipe to search for them, go to YouTube, search for the user name, and subscribe to them before I could find them through YouTube's interface.

ansible 4y ago
Yeah, I noticed this recently. I'd search for keyword 'X'. There are some relevant results in the first dozen results, fine. And then I scroll on because I know that there is more, but the search results just... end? Like there's only ever going to be a dozen valid results for any search? It is a sad day when I have to flip over to bing or somewhere to do a deeper search, rather than staying on-site.
derefr 4y ago
I assume you also tried searching for リラ? (This is in general a kind of “life hack” — people who speak other languages will often use their own language to title+tag stuff even on otherwise English-language websites; so if you want to find stuff from a specific country, you can often surface higher-relevance content if you put your search keywords in the language that country speaks.)
- JasonFruit 4y ago
  I'm... honestly not sure. I think I got the right word, but I don't know a breath of Japanese, so I might have copied the wrong one. At any rate, the results of trying it just now were seriously underwhelming.
  - derefr 4y ago
    The trouble with Japanese in particular, but ideograph languages generally, is that the lack of explicit spaces makes tokenization nontrivial (you essentially need to parse words out using a model built on a dictionary) — and English-language websites don’t even bother to think about doing this when trying to build their search engines; they just throw things into Lucene or Postgres tsvector and think the problem solved. This results in indexing document titles in these languages as if they were bags of individual characters (i.e. splitting on every character.) Other ideograph languages (e.g. Chinese) aren’t so bad when you do this, because enough information is captured per ideograph that losing ordering doesn’t actually change meaning that much. But Japanese is especially bad, because it often “reverts” to the hiragana or katakana alphabets for long strings of words, while still not putting any explicit word-break markers between saidwords. So you end up indexing a “bag of letters.” You can imagine the uselessness of such results. Or, to put that another way: Japanese-language search “works” on English-language sites when you’re searching a term expressed in kanji. But when the term you want turns out to only have a katakana representation, it’s mostly not going to work. And yes, Google themselves know all this and have solved all these problems for Google Search long ago. Apparently, the relevant tech never made it into YouTube.

ansible 4y ago

Yeah, I noticed this recently. I'd search for keyword 'X'. There are some relevant results in the first dozen results, fine. And then I scroll on because I know that there is more, but the search results just... end? Like there's only ever going to be a dozen valid results for any search? It is a sad day when I have to flip over to bing or somewhere to do a deeper search, rather than staying on-site.

derefr 4y ago

I assume you also tried searching for リラ? (This is in general a kind of “life hack” — people who speak other languages will often use their own language to title+tag stuff even on otherwise English-language websites; so if you want to find stuff from a specific country, you can often surface higher-relevance content if you put your search keywords in the language that country speaks.)

JasonFruit 4y ago
I'm... honestly not sure. I think I got the right word, but I don't know a breath of Japanese, so I might have copied the wrong one. At any rate, the results of trying it just now were seriously underwhelming.
- derefr 4y ago
  The trouble with Japanese in particular, but ideograph languages generally, is that the lack of explicit spaces makes tokenization nontrivial (you essentially need to parse words out using a model built on a dictionary) — and English-language websites don’t even bother to think about doing this when trying to build their search engines; they just throw things into Lucene or Postgres tsvector and think the problem solved. This results in indexing document titles in these languages as if they were bags of individual characters (i.e. splitting on every character.) Other ideograph languages (e.g. Chinese) aren’t so bad when you do this, because enough information is captured per ideograph that losing ordering doesn’t actually change meaning that much. But Japanese is especially bad, because it often “reverts” to the hiragana or katakana alphabets for long strings of words, while still not putting any explicit word-break markers between saidwords. So you end up indexing a “bag of letters.” You can imagine the uselessness of such results. Or, to put that another way: Japanese-language search “works” on English-language sites when you’re searching a term expressed in kanji. But when the term you want turns out to only have a katakana representation, it’s mostly not going to work. And yes, Google themselves know all this and have solved all these problems for Google Search long ago. Apparently, the relevant tech never made it into YouTube.