After a week or so, the fracture
gets sticky and is more difficult to set.
If you are in doubt, make sure you see an orthopedist and get needed care before you have to make this decision. If there is drift and a re-set is necessary, i would think a couple of weeks out is getting to the limit. I defer to our orthopedic colleagues on health tap for a more expert answer.