Can you actually go deaf if you listen to loud music through your headphones all the time?

You bet you can: Loud noise, especially close to the ears, can damage sensitive hair cells, and cause progressive loss of high frequency receptors, and later additional loss of lower range perception. Although lower octave cells are more resistant, and full deafness does not usually ensue completely, the HFSNHL may well accelerate the need for a hearing aide early on.

Hearing loss: There is a strong relationship between hearing loud sounds or music over a prolonger period of time and loss of hearing later in life. Best to shield your ears from such sounds when you are exposed whether young or old. There are other causes for hearing loss but loud sound is a major one.