Carefully... Most modern strollers have the ability to remove the fabric components and easily put them in the washing machine; refer to the instruction manual, or contact the manufacturer for additional information. If not use a wet, soapy washcloth using whichever baby soap you utilize for your 'passenger' and just wipe it down. Avoid home cleaning products like window cleaners, etc.
Don't sweat it. While you don't want a child's stroller to be a pigsty, keep in mind that kids' immune systems grow and mature as they are exposed to their environment. There is a growing body of research suggesting that raising kids in too clean an environment can set them up for asthma and other immune-related diseases down the road. So clean it but don't sweat it too much.
I like steamers. I agree with dr cohen's approach with non toxic simple products. I have had some moms report good success with simple hand held steamers. Sold as a house cleaning tool, this simple device pushes out steam under pressure that can blast off dirt & debris from fabric, metal or plastic & dry off almost immediately. It sanitizes as it cleans without leaving any toxic film behind.