Does dad work outside of the house? Do you spend all day with the baby? Then the baby is excited to see a new figure when it walks through the door because it is "new and exciting" whereas you are the "same old and standard" the baby is used to on a regular basis.
Give it time and it will even out. Don't read into something that isn't even there.