Should I be worried if a benign tumor starts to hurt or twinge?

Possibly. Benign tumors do not spread to distant sites. However they can grow and cause local issues. If the growth begins to cause pain it may be causing local damage due to its size. Recommend that you see your primary md for further evaluation. Even benign tumors sometimes need to be removed.