Is it safe for me to wear non-toric lenses if I have an astigmatism?

Maybe. It might be ok, depending on the amount of your astigmatism and the curvature of your cornea. Your ophthalmolgist can advise you based on tests in their clinic.
It is safe, but your vision may not be corrected as well as it would be with toric lenses.