Wondering if this is something i should see a doctor for/do a virtual visit or something i should self-treat. thanks! ?

Check Up and labs: Start with a visit to a doctor. You need to tell someone more about your headache. Insomnia is a common problem as well. There may be an underlying reason for the headaches that can simply be treated. OK to use Tylenol until seen by a doctor.

Headaches: As I see it you have 2 problems and they may well be interrelated. The headaches are characteristic of muscle contraction tension headaches which are common and may be frequent. Stress management therapy by a psychologist should be helpful and may help your sleeping problem. You could consult a sleep disorder specialist or clinic for more help.