I had a hip arthroscopy surgery four days ago for cam impingement and labral tears. Today, I have numbnessbelow my knee. Is there something to worry about?

Possibly not. Depends some: did the numbness begin 4 days after? Or has it been present since the surgery itself? Did you have any form of nerve block before the procedure? There is risk with the traction used in this type of surgery to put pressure on the nerve in the groin area, but this typically cause numbness in the genital region more so than distal to the knee.
Observe for now. Sounds as though this can safely be observed. Leg is put in traction for procedure in order to distract hip, this also applies traction (stretch) to nerves which can lead to sensory changes post-operatively. In vast majority of cases this is something that resolves with time.
Knee numbness. I would contact your surgeon and relay your concerns. Sometimes a long lasting spinal or a regional block can cause ongoing numbness that should resolve after 24 hours. It may also relate to time of traction; you should also ensure that there is no "pre existing" lower back issues.