Why do ear infections only affect teens and, toddlers, and children?

Not the case. Adults, all though less frequent can get ear infections. Ear infections are normally more common in toddlers and infants, simply because they may get more uris.
Anatomy. Dr raff's answer is spot on. The anatomy of the ear is more conducive to development of ear infections in children. As we age this changes so we are more resistant. Exposure is a part of thye equation, but only a part.

Related Questions

Wondering if it's normal for teens to get ear infections?

Some do some don't. I see more outer ear infections (swimming related) in teens than middle ear disease, but both do occur. Some are more susceptible than others.