What might be causing me to have high diastolic blood pressure at only 24?

Multiple. There could be multiple causes including smoking, diabetes, lack of excercise/obesity, high salt diet. Family history also plays a significant role. Diastolic hypertension is more common in younger persons and should be treated. There are also multiple secondary causes that also need to be excluded. See your doctor asap.