Don't swallow. The danger of magnetic toys is if the child is at an age where he/she can swallow the magnet. Swallowing magnets can cause serious intestinal problems, including perforation. I would refrain from buying toys with small magnets for babies and toddlers definitely.
Yes. For young children who are still mouthing things, toys with magnets can be aspirated. Many magnetic toys also use batteries. The small buttoon batteries can cause GI ulceration.