Having some sweets is ok (unless you're diabetic).
Just like anything else, you need to moderate it with other foods. For example, have something sweet for dessert 1-2x/week and when you do, 1) get a normal-size portion and 2) take your time to savor and enjoy it. Avoid eating sweets for snacks because it's hard to keep track of the quantity. And, of course, follow a diet