I do agree that social stigmas make things worse. But there is nothing "negative, or offensive, or stigmatizing," about the words I used in the context I used them. Now, if I had approached a transgendered person and said, "You are a problem! You are a defect!" I would agree that those statements would be offensive and stigmatizing. But 1)I would never do that because that's not how I view them as people and 2) that is clearly not what I did and you have to twist things and take my words out of proper context in order to manufacture offense.
And sorry, but I really don't see how saying, "Surely we can agree that these people have an issue brought about by an anomaly in their development/brains/gene" is any different from what I said in plain meaning. If you view your job as Internet Word Cop, then have at it but this has grown beyond tedious.