Why do you have to remove the whole testicle?

Cancer. If you have a tumor in the testes you will need testes removed.
Relapse risk. There is usually some cancer cells still lurking in the testicle that can later spread so common practice, especially in a paired organ is to remove the entire testicle.