How fast could invasive ductal carcinoma metastasis?

Depends. Depends on size, grade, location, receptors and many other conditions.
It depends... ...On many things. The longer the disease is present, the higher the likelihood of metastasis. Tumor behavior is predicted by a number of factors, including tumor size, grade, receptor expression and tumor genetics (among others). In summary, invasive ductal carcinoma, by the nature of its invasiveness, can metastasize at any time and may already be metastatic at time of diagnosis.
Highly Variable. The risk of breast cancer growing outside of the breast is most dependent on the aggressiveness of the cancer and the stage at diagnosis; it is less related to the type of breast cancer (ductal, lobular, etc). While metastatic disease may be present at the time of diagnosis, a majority of breast cancer patients never develop metastases.