Listen to your doc
One value for a psa (especially one that is within the range of normal) is not usually diagnostic.
What matters more is a trend. If the value is still 3.1 in 6 months, it is reassuring. However, if it climbs significantly it may warrant a closer look.