If your child had a fever, at what temperature should you worry about calling the doctor?

Depends. Younger children can tolerate fairly high fevers; i would be concerned anytime the cause of the fever is uncertain. Otherwise temps up to 104 can occur with relatively minor infections like colds, flu, etc. If the child is lethargic, repeatedly vomiting, seizing, having difficulty breathing, in significant pain, etc. Should be seen by md.
Both temp and durati. Temperature elevation in absolute concern is very important, and need be treated (alcohol sponge baths), nsaids, but duration, even if the temps are low grade (99-100) that go one for more than a week, is more important in your taking a child do a doc.