If I can jiggle my jaw and reduce the dislocation, why should I be concerned?

Because. It is not normal for our jaw to dislocate in the first place. Every time it dislocates and you pop it back in makes it more likely to pop out the next time. At least get it check out by an ENT doc, to make sure you dont have any significant TMJ issues or otherwise.
Bruxism. Long term of effect of this jaw popping will cause sleep and muscle problems. This problem can be annoying and painful, leading to pain in the jaw region, headaches, neck aches and difficulty opening and closing the mouth.