A friend told me I could put a little vanilla extract on a wet washcloth and let my baby chew on it for teething relief. Is this okay?

Yes. A clean, cold, damp washcloth can be very soothing for a baby who is teething, and a little vanilla will improve the taste. Be sure the cloth is in good condition and not ragged, however; you don't want it to come apart in baby's mouth & maybe get inhaled. Keeping a few damp cloths in the freezer gets them as cold as possible, and gives you the opportunity to rotate them.
NO. There is no evidence that it will be effective. I do seem to remember that extracts are made up with significant amounts of alcohol, and that would be inappropriate.