6 ms·
First three norwegian cities I looked up was given the wrong name. (Plesund (should be Ålesund), Bodi (Bodø) and Tdnsberg (Tønsberg)). So, I think it is safe to
by Quai 11y ago
First three norwegian cities I looked up was given the wrong name. (Plesund (should be Ålesund), Bodi (Bodø) and Tdnsberg (Tønsberg)). So, I think it is safe to say that there are encoding issues in the data set.
- bhaak 11y agoThere's definitely something wrong. Looking at the SQL file, "Zürich" is listed as "Zdrich", Munich is only shown as "Munich" without the German name "München" anywhere, and Cologne is listed as "Cologne" with the wrong "Koln" as alternative name. Doesn't look like a reliable data source.
- seszett 11y agoFrench names have the same problem, with "La Réunion" being shown as "La Rcunion" and "Rhône-Alpes" as "RhAne-Alpes". It's strange, I can't really understand how these accents can get mangled as these particular single letters.
- vcarl 11y agoAlmost looks like OCR errors, especially Rhône -> RhAne.
- bhaak 11y agoI don't think that the reason are OCR errors. It's too consistent for that. I think I also have seen such character set corruptions before, but I can't remember how exactly you can get these particular corruptions.
- marco1 11y agoThey're not OCR errors, unfortunately. Sorry for these problems! We have to investigate!
- maze-le 11y agoIt's 2015 and we still have UTF-8 problems... sigh
- spacemanmatt 11y agoOr this is just a sloppy translation/compilation from better sources.
- marco1 11y agoSorry for the inaccuracies! We'd love to fix all those issues -- maybe with some help from the community :) The goal is definitely to turn this into a more and more reliable data source every day.
- marco1 11y agoThis is definitely a major problem that has to be fixed! Thanks for pointing this out again. The complete build process is described in "SOURCE.md". That's where we got these encoding issues from -- right from the source material. Maybe we didn't open or parse the source material correctly. But we were not able to get them without the encoding errors. We'll love to fix all these issues :)
- lobster_johnson 11y agoInstead of Markdown file, you should consider creating a set of scripts that will fetch, cache and process upstream data into the desired formats. As someone interested in using this data, I really need to be able to easily run the entire build process myself.
- marco1 11y agoOf course, you're right, thanks! This has been noted down as one of the next steps that are required.