Should I be concerned about small bumps between my nose and eyes and also on one side of my nose?

See a dermatologist. This requires a clinical exam. Most commonly, the tiny bumps between your nose and eyes are syringomas (sweat gland growths), sebaceous hyperplasia (oil gland growths) or milia (little cysts). However there are other conditions that can result in small bumps. A confirmation by your dermatologist is recommended.
Probably benign. It depends on how long the bumps have been present, whether they are painful, getting bigger, weeping fluid, etc. May people develop benign skin bumps with age. A complete eye examination is necessary to determine if this is a benign condition.
Bumps on nose. Please see a dermatologist. Many benign conditions cause bumps on the nose, and different causes ususally exist for bumps on the eyelids. Milia (whiteheads), oil gland overgrowth (sebaceous hyperplasia) sweat glands (syringoma) cholesterol or fatty deposits (xanthelasma), adult acne rosacea, and many other conditions exist. See a dermatologist to diagnose the problem and make it's not serious.