Will uti pain go away?

Yes: As the infection is treated and goes away pain should also subside.

Yes needs time : Antibiotics, if effective, often clear the bladder of bacteria very quickly because they're concentrated in the urine. However, the inflammation (irritation) of the bladder walls may take days to resolve and the symptoms like burning with urination, frequent voiding, and pain. If these persist beyond one week or high fever or flank pain develops then contact your doctor.