Should a toddler go to doctor with raspy breathing from a cold?

Yes: Yes, he needs to be evaluated, could be viral but respiratory effort needs to be assessed an other etiologies need be ruled out.

Depends on situation: If the kid is playing robustly, feeding & drinking well & sleeping ok at night, you can probably ride it out. If the kid appears warn down, drops his/her feeds & fluids or runs fever, an evaluation would be worthwhile.