Will giving a 3-4 month old some ice cream cause a child's teeth rotting?

Not likely : Early childhood caries is usually caused by too much sugary foods and drink. Putting infants to bed with a bottle of juice or milk will cause decay also. It is good to brush the child's teeth or wipe the gums with a wash cloth if teeth aren't erupted yet. A little bit of ice cream every now and then is fine.

It can: If you are talking one time than it is not likely, but if you are talking multiple times than yes. Children's teeth are actually slightly stronger than adult teeth because the enamel layer is thicker, but you cannot give a child that young Fluoride to keep that enamel strong . So if you do make sure they have some water after the ice cream to wash away the sugar off their teeth.