How does one know what's the perfect condom size to buy?

One size fits all. Regular condoms that fit over a penis are somewhat stretchable, so they are all about the same size. If after trying a box of regular condoms, one feels that they are much too tight, just try the large ones next time (called magnum). Most people are just fine with the regular size.
Condom sizing. Regular condoms usually work for a girth btwn 4.7 to 5.1 inches. If girth exceeds 5.1 inches (130 mm) may need a larger size condom. If girth is less than 4.7 inches (119 mm) may need a smaller size. See this site for more detailed information: http://www.condom-sizes.org/condom-size-chart/condom-size-chart.