If anxiety is a common side-effect of effexor (venlafaxine) and one is taking it for anxiety, how to know whether to increase or decrease dose at any point?

Gradual process: This is part of the clinical skill involved in becoming a psychiatrist. Basically one learns what your baseline anxiety level is/was -- and follows your progress on the medication. The process would be very gradual, incremental changes in dose -- hopefully leading to relief. Some psychiatrist also use written rating forms in assessing anxiety or depression levels, at each visit.