For how long does one need to be off a certain food in order to determine if one is intolerant to it?

About 3 weeks. Standard practice is to eliminate possible food allergens for 3 weeks to give the immune system time to calm down. You may be able to notice improvements in a shorter time but going a full 3 weeks will give a better response if you try to reintroduce the food.