When should I be concerned about a canker sore? I've had it for a week and it just keeps getting bigger no matter what I do.

Usually Herpes, but! If herpetic, can be treated. However many other causes exist. Fever is common as is prodromal stress. Silver nitrate sticks can help.
Canker. Other sores can look like a canker sore so if it is not getting better and growing, have it checked asap. Are you a smoker?