Can wearing a weaker prescription damage your vision?

No. Just won't be able to see well with them. Sometimes a weaker prescription can be made stronger by tilting or moving the glasses. Also it can improve with a pinhole.