Will my child grow out of their eczema?

Most likely. It is estimated that approximately 15% of children with eczema will continue to have it into adulthood. Majority grow out of it by 10 or 11 years old, or have very mild intermittent flares.
Many improve a lot. Many kids will improve quite a bit as the reach between 3 & school age. Some will continue to have rough skin well into adulthood.