I read somewhere that there was metabolic waste in sperm, how is this even possible? i feel like that would damage semen?

In sperm=no. Are they exposed to some occasionally, some probably are. Thankfully there are millions created continuously and those effected by any exposure don't function well enough to produce conception. The healthy ones swim better and reach the egg. In some ways this selects out the best sperm for the job while the egg is the only one available at the time.

Please dont read unscientific material online as most of it is garbage and leads to worries and questions like yours. Enjoy life without esoteric minutiae.