Does the unmetabolized drug remain in body for years if the side effect from the drug is present after stopping it many years ago?

Not years. Different drugs have different elimination rates, but none remain in the body for years. There are some chemicals like lead and heavy metals that get incorporated in the bone that may stay in the body for year. Other chemicals like DDT and biphenyls may get sequestrated in fat and are not metabolized and can stay in the body for years. See this site for drugs of abuse clearance times.