Will losing this testicle stop me from producing testosterone and "being a man"?

Most likely not. The second testicle (if normally functioning) is able to "pick up the slack" and produce adequate testosterone without a change in your "manhood". However, if there is already an issue with the testicle (such as it is small or has sustained prior injury) then it may not produce normal amounts of testosterone. All things equal though with a healthy remaining testicle, there should be no changes.