Nobody has "black" or "white" skin. I'm no word historian but I'm willing to bet that the association of white/black with positive/negative is far older than the slang usage to refer to skin colors; certainly more deeply baked in the language. I wouldn't even be surprised if the metaphorical meanings informed the skin color usage, due to racism. And you'll never eliminate that metaphor from the language, not without doing excessive cultural violence[0]. If you have a problem with the overlap, better to deprecate "black" and "white" as skin color terms at all.

[0] https://en.wikipedia.org/wiki/Black-and-white_dualism