Length of time. If they don't disappear or keep coming back, something else may be happening.
Change in course. Usually no treatment other than otc topicals is needed. If there is a change in the course like systemic symptoms such as fever; chills; or a change which is inordinate such as size; shape; coloration; pus; or if the "ulcers" last longer than the usual course or continue to re-occur, then it would be a good idea to see your doc. Dr. J.
Depends on severity. No cold sore is fun, but one every few months is manageable. Someone who has at least one or two all the time, however, is more severe and concerning. Also, if you know what triggers them, you may be a candidate for treatment (ie someone who always gets them after dental work due to the stretching at the corners of the mouth). Discuss the frequency and severity with your dentist. He/she can help.