Do I need to avoid all alcohol if I am taking baby aspirin?

Not necessarily. Not necessarily. Alcohol in moderation (1-2 glasses of wine or 1 beer) a day should not cause you any problems. Alcoholism or heavy binge drinking can sometimes lower your platelets (the cells in your body that help clot your blood). Aspirin works by also inhibiting platelets, so together this could lead to bleeding problems.
Not necessarily, but. Aspirin is an irritant to the stomach lining; alcohol is also an irritant to the stomach lining. Taking both can increase the possibility of peptic ulcer disease, ulcers, etc. Additionally, symptoms of peptic ulcer disease may not occur until the condition becomes moderate or severe. See you doctor if you have abdominal pain, increased burping, black or bloody stool or more "indigestion" symptoms.