Can our 4 month old who is taking 3.5 amoxicillin for his ear infection also take 2.5 ml of acetaminophen?

Depends on variables: Acetaminophen products come in various dosing forms, drops /liquid/pill/capsule. The proper dose depends on the strength of the product.I'll assume your 4 month old weighs about 16 pounds & the proper dose would be 80 milligrams. That would be 2.5 ml of the 160/5 liquid. The meds can be taken at the same time.

What concetration?: It's depends on concentration. If 2.5 ml=80 mg - it's too much. You can give once or twice for high fever >100.4 f, but half of this dose 1.2 ml (40 mg) 6-8 hours apart. Remember, that tylenol (acetaminophen) is not recommended under 3 months of age.