After a pimple or small acne fades away, a small rolling scar forms despite the fact that I do not mess with the pimple or acne at all. Why is this?

Scar contracts. Acne lesions form when neutrophils chasing bacteria damage the skin in the process. Scar forms and contracts. Whether or not you mess with the lesions (and almost everybody does), you're going to get these until you start managing your acne. If you've had scarring, the benzoyl peroxide you're using clearly isn't sufficient, and you will probably need isotretinoin. You deserve effective Rx. Go 4 it.