Is it possible for me to be lactose intolerant even though i can have dairy at times?

Yes: Sometimes lactose intolerance depends on the dose you take and the source. The lactose in yougurt is partially digested by healthy bacteria in yougurt. It may not affect you as much as milk which is higher in pure lactose. Drinking alot of milk all at once, can make it worse. Cheese, may be low in lactose if not lactose free- and be better tolerated. Keep a log of what you take, when & how much.