Is there a way to prevent myself from getting polyps?

No. Polyps are caused by overgrowth of cells lining the colon wall. In most cases this is genetically determined. There is some research that indicates certain substances in foods may stimulate this increased growth rate but cuurently there is no strong compelling evidence of what to avoid.
Polyp prevention. There is no definite known factor in polyp production in the colon. Most all colon cancers come from polyps, but all polyps dont become colon cancer. There is a long precancerous phase if they do. The best defense against colon cancer is to have regular colonoscopy, which removes precancerous polyps and prevents colon cancer.