If I have dysplastic nevi syndrome and my brother in law had malignant melanoma do my children have any more risk of getting either one?

Yes. Your brother-in-law's genes won't tell anything about your children. Dysplastic nevus syndrome is usually autosomal dominant, passed to each child on a 50-50 chance. It's no reason to forego parenthood; just you be sure that you and all your kids keep a really close eye on your skin. Glad you're proactive.