I have been having a hard time lately when i am working on a computer and i need to look up from my screen to a screen at a distance (like a projector) that my eyes are not adjusting focus quickly. is this eye exhaustion?

Not exactly. It is recommended that you take frequent small breaks from looking at the computer screen. The break need not be longer than a few seconds in which you look at a distant object. For info: https://www.hse.gov.uk/msd/dse/work-routine.htm

If your on the screen for hours your eyes will be strained and tired. You should see your ophthalmologist for an exam and possible glasses if needed.