How safe is it to wear underwear straight out of the package before washing them first?

Not great: While it probably won't kill you, it's always a better idea to wash brand new clothing prior to wearing it. Many chemicals are used in the manufacture and packaging of clothing that can cause irritant and allergic reactions, typically rashes. I recommend washing all items of clothing prior to first wearing them.

I'd wash it first: While it's not terribly dangerous to wear underwear directly out of a package, some people may develop rashes/reactions to the new material due to chemicals used in the processing. It is always a good idea to wash them first but if you don't have the opportunity to do so in a pinch, it probably won't hurt you.