This is a tricky problem in pretty much the entire consumer-facing ML space. For example, even if you don't feed in 'protected classes' - you can say 'the algo

This is a tricky problem in pretty much the entire consumer-facing ML space. For example, even if you don't feed in 'protected classes' - you can say 'the algo finds a way'. It might figure out a certain zip code won't pay back a loan, so it will down weight them accordingly - even though that zip may predominantly be one ethnicity. Is that discriminating on a protected class? Well, kinda. It's discriminating on a proxy for a protected class which, if we're getting technical, is really a (poor) proxy for cultural, genetic, and socio-economic backgrounds. So it's in some ways 'more fair' but it does take away our ability to 'socially engineer' certain decisions by prohibiting use of one feature or another.