When can I feed my baby spicy foods?

6 months. There really isn't any harm in capsaicin (chili peppers) or allicin (garlic) containing foods even at earlier ages, but starting slow will minimize any discomfort during adaptation. Mustard and horseradish may actually be slightly toxic, however.
60 months. It is not necessary to feed children spicy foods. However, if a child is old enough for kindergarten, he's old enough to try little bits of spicy food (not too spicy!) and tell his parents what he thinks.
Six months. When you start your baby on solids, you will give him small amounts of table food that you have prepared, often using herbs and spices that you cook with every day when you feed yourself. There's nothing wrong with letting the baby explore these taste (cultural) varieties. Take it one little step at a time; giving anyone large amounts of pepper or hot spices is never a good idea.