When should ice or heat be used for hand joint or muscle/tendon pain?

Heat up, cool down. Underworked stiff tendons respond to heat such as in the am after sleep or after sitting. While too much over doing it and inflammation needs cold to calm down. Its actually pretty simple. Think of an athlete.
Ice and heat: when? Typically ice is used for the first 48-72 hours to limit inflammation and the associated swelling. Thereafter, when the inital inflammation has resolved heat may used to enhance circulation and improve healng.
Ice for inflammation. Heat for muscle tightness. Heat and stretching before activity, ice after.