If I eat/drink a little dairy, nothing happens, but if I eat too much, I get gas/diarrhea, is this an allergy?

Lactose intoleranc. No, this probably is not an allergy, but rather a common problem with the digestion of the sugar in milk called lactose. A digestive problem called lactose intolerance often develops with aging. People do not manufacture enough of the enzyme lactase to digest the lactose and symptoms of gas, bloating, diarrhea and indigestion can occur. The more milk the more symptoms.
Lactose intolerance. Most commonly, gas and diarrhea associated to diary ingestion is due to lactose intolerance. Digestive aids contaianing lactase are sold over the counter and are most helpful for this condition. Less likely, it is possible that these symptoms correspond to cow's milk allergy. Allergy testing can rule out this possibility.