Why does nyquil make me sleep better than a shot of liquor when nyquil has less alcohol?

Synergistic effect. Nyquil has other sedating ingredients, primarily a sedating antihistamine, combine that with alcohol and the two are more sedating than either one by itself. People with history of addictions should avoid nyquil. Becareful if your are using it for only sleep, and alcohol for that matter. Alcohol actually makes sleep quality worse.
More than alcohol. Nyquil contains antihistamines and dextromethrophan, both of which are very sedating, and this sedation is augmented by the relatively small amount of alcohol present.