God made it so
Many plants -- including grapes, tree bark, and tea -- contain tannins.
These are the plant's self-protective compounds that counteract bacteria
by interfering with their surface proteins. They also discourage herbivores from eating them, with that "puckery" effect on the mouth plus interfering with digestion. Adding milk to tea allows tannins to attack the milk, rather than your mouth.