Why do my ears hurt so much when I try to swim to the bottom of the pool?

Pressure change. The pressure of the water against the air in the outside of your ear drum becomes so high it causes pain. Holding your nose and blowing air into the middle ear equalizes the pressure between the outside of the drum and inside instantly stopping here pain.
Maybe ETD. Maybe eustachian tube dysfunction - not being able to clear ears ("pop" them open) when diving. Divers need to clear your ears every 5-10 feet of diving to prevent ear pain and damage to ears. Could also be a hole in ear drum. Be careful that you do not dive if you notice drainage or blood from ear or have vertigo ("spinning") during or after diving.