I know neoprene is considered a toxic chemical, but is it toxic to use neoprene dumbells for a workout? if so, how is it toxic through using dumbells?

Not likely: Not likely to be enough absorption through your hands to be of any concern.

Probably safe: Neoprene is usually used for stretchy materials like laptop sleeves & wetsuits. It can "outgas" toxic chemicals and when glued to other materials the glues are toxic too. Chemicals in neoprene can irritate skin. But once neoprene dumbells are a few months old, most of the toxic chemicals that might outgas from them have dissipated and there should be no significant hazard from using them.