Filter water but afraid of getting weak teeth. What is best for kids?

Fluoride and more. There are multivitamins with Fluoride available, Fluoride rinse (for kids old enough to reliably spit it out), topical Fluoride that dentist can apply onto teeth. Also make sure your kids eat a healthy diet, brush and floss regularly, and get regular dental cleanings.
Fluoride. Talk to your dentist about Fluoride which strengthens teeth and reduces incidence of cavities by a whopping percent. You may need to provide Fluoride supplement.