Why don't men want sex some of the time?

Same as for Women. No one wants sex all the time, well, almost no one. Many factors affect libido including attraction, adequate sleep, concurrent worries or anxieties, and, among others, how one feels about oneself at any given time.