Four year olds live in the realm of magic (see selma freiberg's book the magic years).
What they think or see in their minds is real for them. There is no textbook answer to your question. You base the answer on what happens. Simply try speaking up for the object that he is talking to and see whether he enjoys the response. If not, you can stop. If he does, keep on going. It should be fun!