5 ms·

>A significant number of people generate CSV files from Excel and if these people upload these files into my application and I assume an incorrect encoding...

>A significant number of people generate CSV files from Excel and if these people upload these files into my application and I assume an incorrect encoding... What if you just gave them instructions?

nkrisc 2y ago
They’ll probably switch to the application that does it for them and just works, instead of the one telling them to do something they don’t really understand.
- unclebucknasty 2y ago
  Selecting UTF-8 from a dropdown on the export doesn't seem too onerous an ask. If that's the differentiator between you and your competitors, then you might have bigger problems.
  - nkrisc 2y ago
    I have sat in on and observed many in-person usability sessions with various applications and websites. In can tell you right now anyone reading this site is in a completely different universe of tech competency than the general public, and even professionals who aren’t tech-focused. You would have lost many of them simply with then jumble of letters “UTF-8”.
    - unclebucknasty 2y ago
      Yeah, I started writing software a "while" ago. I've encountered a handful of users (and charsets) in my time. With CSV exports, there's already a good bit of training you have to do WRT the column layout, file format, headers (or not), cell value formats, etc. There's far more lift involved in training users there than ensuring they select "UTF-8" when they select CSV. And, there's really nothing technical about it, as they don't need to understand what "UTF-8" actually means any more than they need to understand what "CSV" stand for. It's a simple ask in a list of asks. I weigh this against having every developer now believe they have to check/convert every character set, which can be unreliable and produce garbage, BTW. And, speaking of garbage, there are some encodings that you won't be able to convert in any case, so it would be technically impossible to preserve data integrity without pushing some requirement on the source up to the user. So, it's about tradeoffs. And, again, if asking users to choose UTF-8 is the difference between customers choosing your app or a competitor's, then you probably need to be more worried about that than your charset encoding.
      - EraYaN 2y ago
        Hell we just accept xlsx and xls natively. It sucks but it solves many issues. Also causes some, but you'll never get it out of users' heads that Excel files are not a data exchange format.
        unclebucknasty 2y ago
        Definitely helps. Should be able to detect the charset from the Excel metadata, at least. Of course, if that charset uses code points not available in your app's native charset, then you're kind of back to square one (unless your use case tolerates garbled or missing data).
    - jimmaswell 2y ago
      I hope someday, with improved public education, we can have better users. Unlikely with Gen Z apparently having worse computer skills than the previous generation..
      - ghnws 2y ago
        Everyone knowing about character encodings sounds like a lot of time wasted. Gen Z has skills in what is available and popular today, just like the previous generation had.
  - ghnws 2y ago
    Having worked quite a bit in b2b applications, that is exactly the kind of thing that makes customers consider other options.
    - unclebucknasty 2y ago
      Interesting. My experience is that business users are much more teachable and mindful than consumers, especially when working with data.
carlosjobim 2y ago
I think most people would rather get in their car, drive to the ocean, board a ship and join the navy, and come back after a few years abroad - than following some instructions on how to use the computer.
- unclebucknasty 2y ago
  Pretty funny, and probably some truth to that being a user sentiment. OTOH, they have to follow some process to use the software. For just the CSV export, they already have to ensure column orders, values, formats, maybe headers, etc. Selecting UTF-8 from a dropdown seems like the easy part.
  - lolc 2y ago
    > they already have to ensure column orders, values, formats, maybe headers, etc. My accrued compatibility shims disagree.
    - unclebucknasty 2y ago
      Sorry to hear that. You have my sympathies.
- Joker_vD 2y ago
  Joining the navy and then be obliged to follow instructions on so many things you'd normally never even be ordered to do (because now you're in the Navy, sailor! and there are three ways to do any thing: the right way, the wrong way, and the Navy way, and guess which way we use here in the Navy?), just to not follow some instructions on how to use the computer... is something I can actually imagine some people doing.

>A significant number of people generate CSV files from Excel and if these people upload these files into my application and I assume an incorrect encoding... What if you just gave them instructions?

nkrisc 2y ago

They’ll probably switch to the application that does it for them and just works, instead of the one telling them to do something they don’t really understand.

unclebucknasty 2y ago
Selecting UTF-8 from a dropdown on the export doesn't seem too onerous an ask. If that's the differentiator between you and your competitors, then you might have bigger problems.
- nkrisc 2y ago
  I have sat in on and observed many in-person usability sessions with various applications and websites. In can tell you right now anyone reading this site is in a completely different universe of tech competency than the general public, and even professionals who aren’t tech-focused. You would have lost many of them simply with then jumble of letters “UTF-8”.
  - unclebucknasty 2y ago
    Yeah, I started writing software a "while" ago. I've encountered a handful of users (and charsets) in my time. With CSV exports, there's already a good bit of training you have to do WRT the column layout, file format, headers (or not), cell value formats, etc. There's far more lift involved in training users there than ensuring they select "UTF-8" when they select CSV. And, there's really nothing technical about it, as they don't need to understand what "UTF-8" actually means any more than they need to understand what "CSV" stand for. It's a simple ask in a list of asks. I weigh this against having every developer now believe they have to check/convert every character set, which can be unreliable and produce garbage, BTW. And, speaking of garbage, there are some encodings that you won't be able to convert in any case, so it would be technically impossible to preserve data integrity without pushing some requirement on the source up to the user. So, it's about tradeoffs. And, again, if asking users to choose UTF-8 is the difference between customers choosing your app or a competitor's, then you probably need to be more worried about that than your charset encoding.
    - EraYaN 2y ago
      Hell we just accept xlsx and xls natively. It sucks but it solves many issues. Also causes some, but you'll never get it out of users' heads that Excel files are not a data exchange format.
      - unclebucknasty 2y ago
        Definitely helps. Should be able to detect the charset from the Excel metadata, at least. Of course, if that charset uses code points not available in your app's native charset, then you're kind of back to square one (unless your use case tolerates garbled or missing data).
  - jimmaswell 2y ago
    I hope someday, with improved public education, we can have better users. Unlikely with Gen Z apparently having worse computer skills than the previous generation..
    - ghnws 2y ago
      Everyone knowing about character encodings sounds like a lot of time wasted. Gen Z has skills in what is available and popular today, just like the previous generation had.
- ghnws 2y ago
  Having worked quite a bit in b2b applications, that is exactly the kind of thing that makes customers consider other options.
  - unclebucknasty 2y ago
    Interesting. My experience is that business users are much more teachable and mindful than consumers, especially when working with data.

carlosjobim 2y ago

I think most people would rather get in their car, drive to the ocean, board a ship and join the navy, and come back after a few years abroad - than following some instructions on how to use the computer.

unclebucknasty 2y ago
Pretty funny, and probably some truth to that being a user sentiment. OTOH, they have to follow some process to use the software. For just the CSV export, they already have to ensure column orders, values, formats, maybe headers, etc. Selecting UTF-8 from a dropdown seems like the easy part.
- lolc 2y ago
  > they already have to ensure column orders, values, formats, maybe headers, etc. My accrued compatibility shims disagree.
  - unclebucknasty 2y ago
    Sorry to hear that. You have my sympathies.
Joker_vD 2y ago
Joining the navy and then be obliged to follow instructions on so many things you'd normally never even be ordered to do (because now you're in the Navy, sailor! and there are three ways to do any thing: the right way, the wrong way, and the Navy way, and guess which way we use here in the Navy?), just to not follow some instructions on how to use the computer... is something I can actually imagine some people doing.