Ask him- do you want this food? If he says no ignore it and serve the next person.
If he takes the food and plays with it, say "don't you want it"? Take it away, put it in the fridge and warm it for the next meal. If he complains of hunger later say "well, you didn't eat lunch, you will have to await dinner" no substitutes. Hunger pays off. Do not get angry.