Should I seek counseling for self confidence issues? What if counseling makes me even worse?

Mirror Look. Good counseling is slow and thorough. It will depend in establishing confidence with the therapist. Beware, there will be touchy issues to tackle and this may cause you discomfort. Good therapy should help with your confidence but there's no quick fix to whatever answers you are looking for.
Confidence booster. Effective therapy should leave you feeling more confident because you will learn tools to manage and overcome overwhelming emotions and challenges. The goal of a good therapist is to help the client become more self-reliant and empowered. This does not mean there will not be "dark periods" in therapy, but the tools you learn and use will help you through them.