Swollen labia after delivery. Will it go down?

Yes nearly always. While the swelling may persist for a few weeks it will tend to resolve on its own.