Is physical fitness important as I age?

Definitely. It's helpful for general fitness, maintaining endurance and strength, reducing risk of many chronic conditions (eg, diabetes, heart disease, stroke) that often occur later in life, and it may help lower the risk of dementia due to Alzheimer's disease.
Absolutely. A good start would be to try and walk 30min/day 3-5 times per week. You will also want to add resistance training to help build lean muscle mass and help burn more calories.