What determines a baby's gender?

That is complicated: First, you may mean sex which is about the anatomy on a baby. Whether they have a penis or a vagina. That is determined by the chromosomes of the sperm. Gender is whether a particular person feels male or female and how they express themselves. No one really knows how gender is determined but we seem to be born feeling like a boy or a girl.

See Below: Whether the fetus receives an x or y chromosome from the father. Y = girl x = boy.