Is it unhealthy to start dating this guy when both of our ex's are also together?

Probably not. I have seen relationships such as this prosper. You just want to be clear in your minds that they really are your ex's. Otherwise things could get awkward. Bear in mind that if you two had any relationship pre-dating your "ex-dom" there might be some uncomfortable questions that may arise.
W U ex R together? I don't think it has any think to do with this if u truly like each other .