How do I get rid of exostosis in my gums?

Surgery. Removal of exostosis usually done as a preprosthetic surgery to prepare the mouth for better functional partial denture or complete denture. It is a simple surgery since there is no danger to possible nerve damage. A full flap is raised, bone can be chisel or grind away, flap suture with complete closure. Healing expected without pain within 1 week.
Necessary? Most exostoses do not require removal unless they interfere with the placement of a removable partial bridge or denture. If necessary, they can be surgically removed. Discuss your options and the procedure with an oral surgeon.