How soon could you swim with a broken clavicle?

Individual. Depends on the extent of the fracture, the location and pain. Probably 4 weeks would allow some swimming. These can heal quickly and if "cracked" but not through and through or displaced, you may get in sooner. Do not do any activity unless approved by your doctor. Don't be a pain in the neck and don't swim away from your responsibilities!
Pain free n 6 weeks. Most fractures are healed in 6 weeks. Once the fracture becomes stable, it usually stops hurting. Since there is minimal weight bearing on the arm during swimming, you should be able to swim as soon as you can comfortably raise your arm overhead.