How do I know which bottle nipple will work best for my baby?

If he/she is full. If your baby has a strong suck, can take 2-3 ounces in 10 to 15 minutes, and is full afterward, then that nipple works. You can spend a lot of money on a nipple, but in most cases, it's not necessary. Stick with the basic nipple and have some patience with feeding your baby. Experiment with new nipples if your baby is not feeding well with the present one.
Comfortable suck. If your baby is comfortable when s/he is sucking on the nipple and will take between 7-12 sucks, then take a break it is the right size. If your baby is choking or you see a bunch of milk dripping out of their mouth the nipple flow is too fast and you need a slower flow nipple. If your baby seems frantic and is sucking really hard the flow may be too slow. Experiment and you will find what work.