I've been holding my breath while being around strong smells to the point that I could not hold it any longer have I caused damage to my unborn baby?

No. Unless you are holding your breath for unusually long times (like 4 minutes) it's unlikely your blood oxygen levels are getting dangerously low & even so it would be for such a short time that I doubt it could damage your baby. If the smells are due to toxic chemicals (like those in perfumes/ new paint/ new carpet/ tobacco smoke/plastics) you are doing your baby a favor by avoiding these exposures.