Is it normal for my baby to vimit while belching?

Yes. It is considered a wet burp and often increases in frequency with time peaking at about 4 mo before declining.