Is it heat exhaustion if you're getting bad headaches in the heat?

That's one reason. Quite likely but not the only possibility. High temperature itself can provoke migraine, post-traumatic headaches, hypertensive headaches.