Sometimes. If both bother you you have a cows milk or dairy intolerance.
Sometimes one can eat exotic cheeses without problems with this disorder since the preperation of the cheese may change or denature the part of the milk you are allergic to thus rendering the product safe. Sometimes boiling milk will do the same. There are many proteins and components to milk that you could be allergic to so.