Is sex safe for people with brain damage?

Probably: safe from a physical standpoint, but depending on the level of brain damage, there are ethical considerations, such as the ability to consent. "Brain damage" can be very minor and cause no or minimal cognitive impairment, or it can be severe enough to render someone incapable of making an informed decision or understanding their actions.