Mouthwashes do not necessarily kill all the bacteria
Also, when you say you dip the brush i assume it's just the end with the bristles. The handle of the brush was touched by your wife and most likely has the cold germs on it, too. Why don't you simply buy a brush for yourself? They're not that expensive!