1cm is cut-off
Any unknown lesion that grows, until characterized as benign, is of concern.
Gallbladder polyps that double in size in 6 years certainly raise such concern. Your surgeon is likely taking your age into consideration, as well as citing a cut-off of 1cm for polyp
size at which point the polyp(s) warrants cholecystectomy. Close follow-up by surveillance ultrasounds is recommended at the very least.