Is there even a point in going to the doctor if you have high testosterone?

Yes: A proper and correct diagnosis should be determined. If your testosterone is "too high" because of a hormone producing tumor....You might want to know this. There may be other reasons for hypergonadism and your pcp and/or endocrinologist will be happy to discuss this with you.

How do you know?: How do you know you have high testosterone? If you do, and it's not due to taking androgens, you need to be evaluated by your doctor.