I wrote a web scraper to scrape products from some Vinyl Record Distributors. It is amazing to me how careless (or clueless) people are with various unicode cha

I wrote a web scraper to scrape products from some Vinyl Record Distributors. It is amazing to me how careless (or clueless) people are with various unicode characters. I had huge amounts of rules for "unifying" unicode, so I could then run the result through various regular expressions. It wasn't just hyphens, but I'd run into all sorts of weird characters. It all worked, but was very brittle, and constantly had to be tweaked. In the end, I used a machine learning model, which I wrote about here[1] [1] https://blog.line72.net/2024/07/31/the-joys-of-parsing-using-machine-learning-for-data-extraction/ https://blog.line72.net/2024/07/31/the-joys-of-parsing-using...