Can listening to loud music really cause hearing loss?

Absolutely. It depends on two variables. The intensity of the noise (greater than 85 decibels) and the duration of exposure. If you are exposed to loud music for at least 8 hours a days and if the loudness is great enough, you will develop NIHL, (noise induced hearing loss). And this is a permanent high frequency hearing loss.
Yes. Any persistent loud noises more than 80 db can cause damage to the hair cells of the cochlea and thus cause hearing loss.