Will putting water overnight in the fridge purify it from pollutants / contaminants?

No. Cooling water does nothing for removing chemical or biological contaminants. If the water sits overnight, larger particles may drop to the bottom, but the water will not be purified. For that one needs a filtration/ purification system, depending on what contaminants/pollutants are present. Have the water tested to find out what it contains & then contact experts for treatment.