Liver. Liver which is not healthy can make people hold on to metabolites. Important to have healthy liver to not have problems.
People are different. Some people are more susceptable to drugs, etc. Than others, and so have more side effects. This can be because their metabolism for a particular drug is faster, and more extensive. For example, some people are more sensitive to alcohol than others, so they get intoxicated with very little while others need a great deal for it to take effect. Those who often drink can build a tolerance to it.