My 4 yr olds stool is almost always very loose. Should I be worried?

Not yet. I would evaluate the diet first. Go away from dairy for a while and see if it changes. The biggest factor is "failure to thrive". This means is your child developing well. This is why we use growth charts to watch patterns in children.