I had total shoulder replacement surgery 4 mos ago but still can't lift my arm to shoulder level, should I be worried?

Probably not. Without complications it usually takes 6 mos-1yr to obtain the maximum benefits from shoulder replacement including range of motion and strength.
Possibly. 4 months after surgery it is common to have continued stiffness and weakness of the rotator cuff. If your xrays look good and your therapist can move your shoulder farther than you can without much pain than you might just need more rehab. It can take up to a year to fully recover. If you still have pain then there might be a tear in the rotator cuff. Nerve injury is a rare possibility as well.
Maybe. If arm can be moved passively further than you can move it actively, that is weakness. One possible cause of weakness after shoulder replacement is failure of the subscapularis to heal. This can be detected on examination. This is uncommon, can be very difficult to re-repair, and functional result unpredictable. If passive and active motion are equally limited, this is stiffness. Ask your doc.