Should I be concerned because I have a wicked headache accompanied by nausea?

Yes, you should! If you've never had such a headache before, head to the emergency room. If you have them regularly, it is probably migraine and you still need to see a doctor, but there is no need to go to an er. Otc drugs work for many - I am partial to my migralex, but some people need a prescription drug such as sumatriptan, or Imitrex tablets or injections. Injections are very easy to give to yourself.