Why is deja vu a symptom of brain damage?

Not really. Deja vu is a symptom of temporal lobe seizures which may be from brain damage but could also be from faulty wiring of the temporal lobe that happened spontaneously.