Sex is relationship!
If he has attraction to others and no performance issues, there is most likely a major problem with your relationship.
The mind being the largest sex
organ, attitude is most important. I would have concern that he does not really care much for you and i suggest you avoid having sex with someone like him, who appears to demonstrate significant promiscuity. Move on. I bet you deserve better.