Can brushing the back of your tongue cause the tastebuds to swell up. coz mine don't seem to go down at all?

They may be normal: I would suggest that you have them checked by a physician or dentist to see if they are normal or need to be diagnosed and treated.

Possibly....: Typically the taste buds vary in size and you may or may not be noticing your normal sized taste buds. Please see your dentist for a thorough examination to determine if your taste buds are normal-sized or traumatically enlarged.