Is it true that wiping my baby's infected eye with tea will make it better?

Probably not: There are many home remedies that are often suggested, but most of them to not have evidence to support them. It depends on the cause of the infection. If it is allergic in nature, rubbing them will sometimes make it worse. Most likely the cause is viral and time will help the eye to resolve on its own, but you should go see your pediatrician as they may give you drops to treat the infection.

No: I would not recommend putting tea in any eye.