Yeah, try labeling all blacks from the US, the Caribbean, Brazil, and several parts of Africa "Brown People" and you'll get an earful, if not a mouthful of knuckle sandwich.
Similarly, try that "Brown People" line on people from different parts of India.
Who is refusing dignity to different cultures?
How is lumping people of different cultures "Brown People" any different from the old, "They all look alike," idiocy?