A doctor recently checked my blood pressure at 80/40. Should I be concerned?

Symptoms? If you are not symptomatic then probably not, but i would go have it rechecked.
Not worried. BP is indeed low, but as you do not report any symptoms this may just be normal for you or there may be an error in measurement. If on the other hand you had symptoms related to having low BP this is definitely too low.
Yes. Unless you are taking BP meds & being over treated or dehydrated, this low BP almost always means low adrenal function. Find a doctor at acam.Org to help you.
Bp. The pressure you list is low but could be fine for some folks. Best to discuss with the doctor who knows you and your situation.