Not that easy. It sounds that you are bothered by this but can't really set your boundary. There is no easy way out, but to have a face to face conversation with this new boyfriend to prevent further emotional burden of both sides. I don't recommend do it on the phone, in email or text. These ways can be perceived as less seriouss than a face to face conversation.