Is exposure to household chemicals dangerous for my baby?

Yes. There are different household chemicals, with different hazards. Even water is a drowning hazard. And the danger is not limited to chemicals; honey can be toxic to infants, and small pieces of food can cause choking. Don't fall into the trap of thinking, "chemical bad, natural good." or if you do, at least don't include poison ivy in the "good" category.
Yes. A good way to answer this question is to begin by asking "is this chemical harmful for an adult?" if the answer is "yes" then children's danger will be magnified given the more vulnerable nature of their thin, baby-smooth skin. As such, harmful substances penetrate their skin more readily. Likewise, a similar danger exists if the harmful substance is inhaled in their tiny lungs.
Yes. Exposure to many common households products can be very dangerous. It is extremely important that parents protect their children from these hazards. Chemicals, cleaning supplies and medications should all be kept in cabinets with childproof locks or on shelves out of the child's reach.