Aging. There is a natural decline in testosterone production with aging in men, not unlike decline in estrogen Progesterone in women at menopause. Alcohol, some drugs and some genetic disorders also lower testosterone production.
Low LH. We don't know why, but your brain seems to get satisfied with less and less testosterone over time, so it lowers the output of leutenizing hormone (lh) which is what tells the testicles to make testosterone. Increasing the signal to the testicles fixes the problem, but nobody knows why it happens. My guess is it has to do with evolution, older men may not make as many perfect babies.