Found a tiny lump inside the back wall of my vagina, under the skin. After feeling it, it seemed to "dislodge" and move higher up. Cause for concern?

Unlikely. Sometimes you can feel a collection of stool through the back wall of vagina. This would be consistent with the ability to "dislodge" the ball. Check for presence after your next bowel movement and if gone then no worries. If still present need to see gyn for exam.