I'm 11w3d pregnant, I refused to have nuchal translucency test. If baby had down I would keep it anyway. My ob/gyn was so perplexed. Am I wrong?

No. Such testing is really about how much you want to know. Occasionally, the results of such testing can have management implications, usually not. As a rule, I avoid doing testing if I don't know what I'm going to do with the results. Sounds to me like you're one step ahead regarding the non-significance of the results. Good luck.
Your choice! I believe strongly that the choice is 100% yours. The key here is that NT testing should be "offered" to all pregnant women. "Offered" means the woman may choose. Many doctors (and many non-doctors!) do not understand why a patient would decline a test; that doesn't mean they judge you badly. Good wishes.