Will using the gym's rowing machine make my upper-body look like olympic swimmer?

Not quite. It can make you look more like a rower (as opposed to swimmer) http://www.Carlosdinares.Com/wp-content/uploads/2012/01/aus-quad-2011-wc.Jpg.
Doubtful. Although a rowing machine may provide a nice workout to the muscles of the upper back and around the scapula, only a minority have the time and genetics to produce highly-developed musculature. So, do the workout for the strength gains, overall health benefits and stress reduction, but not necessarily to look like the very gifted olympic athletes.