Should I self treat itchy, red bumps on toes?

You could. It sounds like you have the beginning of athletes foot. Try an antifungal for a week. If it does not help see your podiatrist.