Would a physical at a physical therapy clinic be different than at a doctor's office?

Should be same. Physical therapy should not be inherently different based solely upon location of the treatment.
Could be. I'm not sure exactly what a physical at a physical therapy clinic would entail, but a physician would go over your family, social, and medical history, medications, allergies, immunization status, preventative health and what is recommended for your age and risk factors. They would also examine your various organ systems. A physical therapist is not trained to conduct a physical like this.