Can endometriosis of the ovaries cause permanent infertility in a woman?

No. Ovarian endometriosis (called endometrioma) is associated with infertility but some women will not have a problem. We try not to remove the ovarian endometiomas these days because we may make the fertility potential worse. Medical therapy usually will not make them go away if they are over 1-2cm. Discuss your situation with your rei for options and stay positive.
No definitely not. The endometrioma is no different than endometriosis in terms of infertility. If you are at surgery, proper treatment techniques can be effective. The fertility effect is much more based on endometriosis and age of the female than even extent of disease. Maybe severe stage IV endo results in faster decline in fertility. The endometrioma is not all that important in and of it's own.