Can 4-5 coffee be the cause of me having a difficultly sleeping?

YES!: Cut down or try decaf. If you must have coffee try not taking any after 3 pm or even earlier if this does not help.

Caffeine. : Coffee contains caffeine, a potent stimulant. If it's the taste you like, try to substitute your last 2-3 cups with decaf and see if you sleep better. Or if it wouldn't disrupt your routine much, don't drink those later cups at all.