In my first linguistics college course, the professor discussed how some words take on negative meaning over time. (This is called pejoration.) Some terms for ethnicity and special needs have changed to different terms every few decades. My professor said these terms change so often because a culture hasn't changed their negative thinking toward an ethnicity, race, or someone developmentally delayed.
I've come across this when I have not used the politically correct term, and all the sudden I was under attack. I never meant harm, but I was "wrong" because I wasn't being politically correct.
Really, it's the speaker's intention that matters more than using the politically correct term. (Some terms will almost always be wrong.)
So we shouldn't jump on anyone's case if they use the wrong term innocently. Only if they mean harm. We need to work on creating positive associations for those with special needs, races and other cultures, not change the terms every few decades!