If you have an allergy to latex and use a latex paint in your house, are you allergic to the walls?

NO!!! Latex paint does not have any rubber latex. It is called latex paint only because it looks like latex when it dries. Fumes from any paints are irritating when painting and soon after paint has dried. Especially to those with other respiratory problems like asthma.