Would too much calcium supplement cause feet and legs to retain fluid?

Possible. You would have to take massive amounts of calcium to cause this because you can not absorb much more than 500mg at a time. The rest would be passed out in your stool. However, some calcium supplements have lead contamination which can cause kidney problems and thus fluid retention. Be sure that any supplement that you take is independently tested for contaminants (most are not.).
It could. High levels of phosphate (hyperphosphatemia) or low levels of phosphate (hypophosphatemia) can effect the body: calcium and phosphate have to be in balance. If you take thyroid medications, make sure you do not take calcium supplements together as it might interfere with thyroid medication, and swelling is a complication of hypothyroidism.