Can your hymen not break but just gradually go away as you have more and more sex?

Yes possible. Hymens can be very variable. Opening can be very narrow, so large that it is little more than a circumferential ridge and anything in between. Hymens with large openings mat never break, and certainly wden or seem to go away as you have more & more sex.