Very few need... Very few need swaddling in the first place. If you are doing it to keep the child from scratching herself accidentally, they should definitely be over that need by 3 months at the latest.
As soon as you can. The initial swaddling of an infant helps them transition from the tightness of the womb to the freedom of outside. It is comforting/the feeling they remember.As soon as you can loosen the wrap and they settle okay, remove the blanket.We lose babies every year who suffocate in blankets, so the practice can be dangerous.
3-4 months. This is generally when the 'startle reflex' wanes in infants so they arent startled awake as easy and when baby's awareness level allows them to soothe themselves without the swaddling. Most babies can stop swaddling by 4 months - if they are swaddled at all.