The context you're missing here is that "Black culture" in the U.S. really means "a blend of many African cultures of only poorly-known detail, plus a dose of s

The context you're missing here is that "Black culture" in the U.S. really means "a blend of many African cultures of only poorly-known detail, plus a dose of some parts of western European culture, as heavily mediated by slavery and discrimination". It's associated with skin color most predominantly because that's the connecting factor for most people who were stripped of their original cultures and forced into slavery.