Are there any permanant side effects when taking mirtazapine?

No. Mirtazapine has a 20 40 hours of half life and once it is discontinued and out of your system, it should not have a permanent side-effect.