Did some minor touch up paint on holes in the wall and allowed 5 hours for it to dry before my pregnant wife came home. Should I be concerned?

Depends. If you used latex paint, there would be no problem because it is water based and does not contain any solvents that could affect an unborn baby. If you used an oil based enamel paint, even with good ventilation, your wife should avoid the area while there are still fumes in the air.