3 weeks. A fracture on the tip of the finger wont require hardly any time to heal, and will be sore while you use your hand. On the other hand (pun intended), a fracture closer to your wrist may take 3-4 weeks before safe to recover use of your hand. It may even require surgery, which can accelerate healing and minimize immobilization (read immobilization as future stiffness). If broken, see a doctor.