Inverse correlation in the US doesn't necessarily mean it's not a factor. You could say for example, that less educated people and therefore statistically poore

Inverse correlation in the US doesn't necessarily mean it's not a factor. You could say for example, that less educated people and therefore statistically poorer have more children because less access to contraception/abortion, and richer people want to be sure they can afford to send their kids to university (all of this needs to be backed by actual figures, I'm just making random claims) so will have fewer kids. I'd say you need a different kind of measure to say this, at least by comparing people within the same social and economic strata.