I still think the meanings are distinct. For instance, you could say: "I can't get a home loan because I'm poor", "I qualify for food stamps because I am poor"

I still think the meanings are distinct. For instance, you could say: "I can't get a home loan because I'm poor", "I qualify for food stamps because I am poor", but attributing these things to being broke sounds a bit funny. By the same token, you wouldn't say "My grandparents grew up broke, but became middle-class"