My doctor says I have pre-barrett's esophagus, how worried should I be?

Have doctor clarify. There is no such diagnosis as pre-barrett's, therefore you should have your doctor clarify what he/she specifically means. In most instances, they are indicating the presence of reflux esophagitis or hiatal hernia.
Close follow up. It means that the lining in you lower esophagus has been damaged usually by the the acid and/or bile in your stomach. A severe form of barretts is associated with developing cancer. Therefore you should follow your doctors advice ( reduce weight, take anti acid medication, and be followed with endoscopy closely).