Was it the wrong thing to do by not allowing the doctor to do a rectal check to see if there was bloody stool?

Yes, but...: Unless you have a polyposis syndrome, risk of malignant colon polyps at 41 years old is low. In older patients, the vast majority of polyps are non-malignant. Polyps may bleed, but at 41 the possibility of juvenile retention polyposis is low. However, bleeding per rectum at your age more typically runs the spectrum from hemorrhoids & anal fissures to crohn's/colitis, ischemia, & infection.