Is headache a normal secondary effect of nuvaring when removing it every month?

Yes. This is very common. Withdrawal from estrogen is responsible for headache, and sometimes migraine, in many women. The timing you describe coincides with the drop in estrogen that occurs when you remove the ring to have your period. The same thing occurs for many women as they start the placebo pills when taking ocp's. It is treatable with a low dose estrogen during the time you have the ring out.
Nuvaring Headache. Nuvaring and other hormone contraceptives are associated with headaches in some people. The timing of your headaches are a bit unusual. If the headaches are not severe, treated effectively with tylenol (acetaminophen) or Ibuprofen like meds, and the NuvaRing works well for you otherwise; then i would not worry and treat the headaches. You can try other bcp to see if you react better. See your doc.