Can toxoplasmosis be fatal?

Yes. For most healthy people, toxoplasmosis is not fatal - in fact, 80-90% of healthy people don't ever get symptoms of the disease. However, if you are pregnant or are immunocompromised (aids), you are at risk from complications including death. That being said, death from toxoplasmosis here in the U.S. Is still uncommon (about 750 people die each year that we know of).
Not for adults. Toxoplasmosis can cause congenital anomalies (birth defects) but it will not harm any adult who has an intact immune system. Do not change the litter box or put your face to a cat's if you are pregnant. Cats unlike dogs feed on live animals (mice, birds etc.) and that's how toxo gets into the cat and from the cat to humans. This is why you should eat well-cooked meat.