How come there aren't many elderly adults with birth defects?

There may be: Some major birth defects result in shortened life-spans, which is why you don't see elderly adults with these conditions. However, many major birth defects can be corrected, at least partially, so that the problem is no longer present in the older child. As for minor birth defects, they are common, and you can find many elderly adults with these; they are just not obvious to the casual observer.

Birth defects: Well birth defects are the number one cause of infant death. So many people with birth defects don't survive childhood. But more adults than you think have birth defects. And better medical care for children with birth defects helps them to survive longer.