Have you heard of a baby being born with teeth? how rare is that?

Yes: Yes, babies can be born with a tooth. It is not very common but it does occur. When they are born with a tooth most often it is removed as it can cause problems feeding and usually they will get another tooth in its place at the appropriate time.

Can happen : Neo-natal teeth, extra teeth that appear @ about the time of birth, can interfere with breast feeding and may need to be removed. Must be careful not to remove or harm the real baby teeth. Ask your pediatrician for referral to a pediatric dental specialist.

Yes: I've seen about half a dozen in 3+ decades as a pedi. The loose ones would need to come out because they could be inhaled into the lung. ( cause problems) The firm ones need to be left alone as they will be joined later by other primary teeth.