You should not be worried yet at this time.
But, you should definitely talk to your doctor. There are many things that can cause low white blood cell count- from drugs, infection, autoimmune problems, congenital, vitamin deficiency
, malignancy etc. Your doctor should do further evaluation. If needed, your doc can refer you to a hematologist ( blood specialist).