Based on my past role, you can't even assume UTF-8 when the file says it's UTF-8. Clients would constantly send CSV or other files with an explicit BOM or othe

Based on my past role, you can't even assume UTF-8 when the file says it's UTF-8. Clients would constantly send CSV or other files with an explicit BOM or other marking indicating UTF-8 but the parser would choke since they just output native Windows-1252 or similar into it. I think some programs just spit it out since it's standard.