My husband is a police officer who works these types of crimes.
Patting and especially resting their hand or your thigh is considered sexual harrassment especially since it made you feel uncomfortable enough to question the action. Teachers are trained on appropriate touching. Since this made you feel uncomforable then you should tell a trusted adult. It's never too late to tell someone.