Where? The body has several areas where yeast are normally found. It represents about 5% of the dry weight of poop in most people at any age. It will form superficial infections on the surfaces of the skin & inside the mouth. None of these cause brain dysfunction in babies. If it enters the blood of an immune compromised patient it can cause a fatal meningitis & blood poisoning. This is rare outside an ICU.
Yeast where? Commonly babies may develop "thrush", a yeast infection in the mouth due to candida species. There are other yeast-like fungu which can invade the central nervous system, including, among others, cryptococcus, coccidiodes, blastomyces, etc.