How frequently do I have to be exposed to passive smoking before it becomes life threatening?

Nobody knows. I am unaware of any studies that look for a "dose response curve" for second hand smoke. Just as with smokers themselves, the exposure probably needs to be daily or near-daily for decades, so it is unlikely that lasting damage (chronic bronchitis, cancer, heart disease) could occur with short-term or low-level exposure, but we simply do not know what is safe. Best to avoid as much as possible.