What are the consequences of removing an ankle cast few days early so I can go to camp?

Probable none. I sure the doctor will make sure there is adequate healing before removing the cast.
Depends. Depends why it was put on sprain? Break? Post op. Truth is the patient should not be the one removing it and if the doctor is the removing it then you have nothign to worry about.