3 ms·

Thanks for the great work! Btw If I may ask, when you compiled the Institutions, did you apply some disambiguation? For example consider that UCLA === Universi

by eunos 6y ago

Thanks for the great work! Btw If I may ask, when you compiled the Institutions, did you apply some disambiguation? For example consider that UCLA === University of California, Los Angeles

ck_one 6y ago
Great question! A combination of techniques were involved in cleaning up institution names. First, I deleted all department names so that only the university/lab name was left. Second, I used local sensitive hashing (wiki: https://en.wikipedia.org/wiki/Locality-sensitive_hashing https://en.wikipedia.org/wiki/Locality-sensitive_hashing) to find similar strings. LSH maps similar strings to the same bucket. This resolved some ambiguity like: UC Berkeley/University of California Berkeley etc. But there was also a good chunk of manual clean up necessary at the end.