5 ms·

Just wondering, is there a source listing good practices when it comes to implementing how one should sanitize real-world, subjective fields like human names, g

Just wondering, is there a source listing good practices when it comes to implementing how one should sanitize real-world, subjective fields like human names, gender, race etc.? And other gotchas while implementing similar things?

bombcar 6y ago
https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... Long story short use a text field that takes UTF-8.
shakna 6y ago
> sanitize real-world, subjective fields like human names, gender, race etc.? The simple answer is _not to_. Use something you can stuff a bunch of arbitrary bytes into, that will fit the byte encoding that the system is currently using. Don't sanitise it. Have no expectations about what it can contain. Have a length, have some bytes, and don't ever dare to try and look inside it.
- hamilyon2 6y ago
  Depends on what kind of software do you write. When dealing with anything near banking and human life and anything regulated really, I strongly suggest heavily sanitizing all of inputs. No control characters, no invalid utf-8, no weird whitespace and white list characters you want to allow. Actual list is domain-specific, of course. Limiting input field to reasonable 20 characters will help defence in depth too.
  - shakna 6y ago
    And that's how you end up with a bank unable to handle the existence of certain people because their legal names exceed character limits, contain unexpected characters, or surprise the programmer in other ways. The legal name must be an exact representation, and appear identically. Sanitising it will break things.
ufmace 6y ago
I'm not sure there really is a way to have universal good practices, since it really depends on what you're using the information for. For ex - why are you collecting gender? If it's for some kind of medical process, the requirements of that will tell you what you actually need to know. If it's for some kind of dating thing, you can also figure out from that what you really need to know. If you don't actually know what you need it for, then maybe just don't collect it at all, and you have nothing to worry about. Ditto most other requirements. Do you need a name for a user in order to meet some kind of legal requirement, or do you just want to be able to put "Hello Username" on the page somewhere? Things tend to get sticky when you might have to interact with multiple external systems with different requirements. Maybe you help run people through some kind of legal process in various countries, and country A's system doesn't support people having a middle name, but in country B this is common and necessary and their systems require it.

bombcar 6y ago

https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... Long story short use a text field that takes UTF-8.

shakna 6y ago

> sanitize real-world, subjective fields like human names, gender, race etc.? The simple answer is _not to_. Use something you can stuff a bunch of arbitrary bytes into, that will fit the byte encoding that the system is currently using. Don't sanitise it. Have no expectations about what it can contain. Have a length, have some bytes, and don't ever dare to try and look inside it.

hamilyon2 6y ago
Depends on what kind of software do you write. When dealing with anything near banking and human life and anything regulated really, I strongly suggest heavily sanitizing all of inputs. No control characters, no invalid utf-8, no weird whitespace and white list characters you want to allow. Actual list is domain-specific, of course. Limiting input field to reasonable 20 characters will help defence in depth too.
- shakna 6y ago
  And that's how you end up with a bank unable to handle the existence of certain people because their legal names exceed character limits, contain unexpected characters, or surprise the programmer in other ways. The legal name must be an exact representation, and appear identically. Sanitising it will break things.

ufmace 6y ago

I'm not sure there really is a way to have universal good practices, since it really depends on what you're using the information for. For ex - why are you collecting gender? If it's for some kind of medical process, the requirements of that will tell you what you actually need to know. If it's for some kind of dating thing, you can also figure out from that what you really need to know. If you don't actually know what you need it for, then maybe just don't collect it at all, and you have nothing to worry about. Ditto most other requirements. Do you need a name for a user in order to meet some kind of legal requirement, or do you just want to be able to put "Hello Username" on the page somewhere? Things tend to get sticky when you might have to interact with multiple external systems with different requirements. Maybe you help run people through some kind of legal process in various countries, and country A's system doesn't support people having a middle name, but in country B this is common and necessary and their systems require it.