What is the best non-prescription thing to use for sinus headaches?

Several options: Try Acetaminophen or ibuprofen*. Since sinus headaches may be due to the sinus pressure, decreasing the pressure will help- an anti-histamine drug ( eg diphenhydramine, loratadine) along with a decongestant (pseudoephedrine) is a common combination. *several otc products have a combination of these ingredients, but make sure you check to see if they interact with your other medications.

Sphenoid sinusitis: Most what is called sinus headache may not be related, unless there is discharge and localized pain oversinus over the counter prescription like decongestant, antihistamine and mucolytic are fine for just brief period---persistant headache with sinus inflamation need antibiotics--remember sphenoid sinus if untreated infections spread to cerebral venous sinuses.