A quick phone call.
Personally, i wouldn't get peeved, but it does help to leave a phone message or a call letting us know who you're seeing and for what.
The specialist might ask us for your medical records, recent labs, imaging
, meds, etc, and we want to be as helpful as we can. Many offices are "fragmented" and communication between them is poor, so it's good to try to keep everyone in the loop.