Is it normal to have pain after cast removal for wrist fracture?

Yes. It is normal for a wrist to be very stiff and sore when the cast comes off. It can sometimes last for 3-6 months but does typically resolve. Occupational therapy can help with the stiffness.
No. Although there be some discomfort and atrophy secondary to disuse severe pain is not expected. This may be a resulting from nerve pain. Talk to your doctor.