Babies, like adults, can have cold extremities
simply because of the environment.
Cyanosis, or a blueish color, to the fingers or hands can be a little more concerning - though not always. I wouldn't be too concerned unless there is consistent cyanosis or a curious change/decrease in finger/hand movement. Use mittens and if you're really concerned, talk with your doctor.