Why didn't they fall out? Are there missing permanent teeth? Are there impacted permanent teeth? What's the condition of the roots of the baby teeth? Know that if a tooth is lost adjacent and opposing teeth will collapse into the space provided, creating an occlusion problem called posterior bite collapse.
Please have a dentist check what's happening and advise you.