3 ms·
I don't know exactly what Yelp does, but assuming that they have listings for restaurants that aren't their customers, one cause could be that they take in data
by Someone 10y ago
I don't know exactly what Yelp does, but assuming that they have listings for restaurants that aren't their customers, one cause could be that they take in data from lots of sources. Ideally, that is all in the same format, with an enforced data scheme.
However, if you require that, you'll notice that very little data manages to make it through your entry port. Few suppliers will want to bend their system for you to comply with your rules, and those who do will find inconsistencies in your specs or just ignore them (different encoding, not sending state abbreviations separately from city names, forget to encode HTML entities, encode them twice or thrice (my personal record find here is four levels of HTML encoding in data stored in an excel file), etc)
So, you end up with quick and dirty hacks that convert the suppliers format (or, rather, what you _think_ it is, as the supplier won't be able to tell what their format is, either) to yours, fix some egregious errors, etc.
Hundreds of data suppliers, code with tables mapping their codes to yours, or that map known errors in their input to corrections, 'smart' programmers who notice that they can replace that 10,000 entry table with corrections with 20 lines of code and a 1,000 entry table, except that, a few months on, that 20 lines have ballooned into 1,000 that nobody understands, so that this code can't be used for handling the data from a new supplier anymore, and the line count starts increasing rapidly.
On top of that, once you operate world-wide, you'll learn the joy of differences in addresses. Does a country have states? Zip codes? If so, where does one specify them in an address? If you want to localize that in your app (in yelp's case, people may want to show an address to a taxi driver. For that, it would help if the address followed local conventions) line count skyrockets.
- miguelrochefort 10y ago> On top of that, once you operate world-wide, you'll learn the joy of differences in addresses. Does a country have states? Zip codes? If so, where does one specify them in an address? If you want to localize that in your app (in yelp's case, people may want to show an address to a taxi driver. For that, it would help if the address followed local conventions) line count skyrockets. This is not the first time this issue has been faced. Why would they reinvent the wheel instead of just using libraries and conventions?
- vertex-four 10y agoYelp would likely wind up forking such a library, if they used one, when it shows bugs.
- bpicolo 10y agoThere's a layer on top of a number of i18n tools at Yelp, yep.
- Someone 10y agoThis is something that grows slowly on you, and it is a very hairy problem. By the time you realize the scope of your problem, finding a library that can be shoe-horned into your code and doesn't have huge regressions on your data is hard. Also, assuming a robust generic solution exists, it will almost certainly be slower than a customized one. It is very tempting to think "but our data is relatively clean, we don't need the full feature set of that library that makes imports take an hour longer to run".