Close to vital organ
If the bullet is superficial and easy to remove is one thing, but if it is deep and close to vital organs.
etc...Cutting and fiddling to remove it may do more harm than good. Often, the bullet is "scarred/sealed in" a normally does not cause much problem...So, if the doc/surgeon believes it is too risky to remove, leave it alone.
consult your doc..