Can you tell if you have decayed tooth by the smell?

No. Most decayed teeth have no smell. However, if the decay is really deep infecting the nerve of the tooth, the resulting infection ( abcess) can definitely have a smell.
No. As mentioned above a cavity typically does not have a smell. If you are experiencing a foul smell, it is likely a sign of an infection. Often this is a result of an acute abscess or chronic periodontal disease. Ther are many other causes of halitosis...Bad breath...So it is best to see a dentist for an evaluation.