Could be many things
Making eye contact
opens a person up and that makes some people uncomfortable.
There are endless possibilities why so guessing is not a good idea. The best way to find out is to ask but ask in a way that is not accusatory. Since the word "refuses" is used i wonder if you are frustrated that the patient is not listening to you. Ask the patient if they know they are not making eye contact with you.