How to figure out why my memory is so bad?

Generic thoughts: You provide no additional information, so will provide a list of potential problems: medication adverse effects, sleep disorder, emotional distress, vitamin deficiency, mass lesions in the brain, thyroid problems, chronic infections, (like HIV and syphilis), prior small multiple strokes, prior multiple head injuries, family history of a dementia, etc. A neuropsych test profile might assist.

Hard to say: as other doctors have mentioned more information necessary to be able to better guide you. as this could be related to something as simple poor sleepor as complicated and rather unlikely as early onset Alzheimer's

Other conditions: that cause dementia or memory problems,in addition to those listed by Dr. Mechanic, include a solitary lacunar infarct in the anterior nucleus of the thalamus, Normal Pressure Hydrocephalus, memory disorders associated with Epilepsy, Parkinsons Disease and related disorders,, a variant of ALS and a host of childhood degenerative conditions like Tay Sachs Disease, and Neuronal ceroid lipofuscinosis

Primary vs secondary: Possible reasons endless & range from easily fixable to requiring specialty care. Taking lots of meds? Dementia in the family? You may need full work-up to include formal cognitive testing & brain CT/MRI. Meantime, avoid alcohol, take vitamins (or even better eat foods) containing niacin, thiamine, folic acid, pharmaceutical-grade omega fatty acids, vitD & magnesium. Check out www.lumosity.com.