If neither of your parents had colon caancer or polyps, you are probably not at increased risk based on grandfathers history and can wait til age 50.
But if either parent had polyps or cancer
consider earlier screening with a colonoscopy
. This all assumes your are not having any symptoms now.