Are there any downsides to constantly going to bed late, but getting enough sleep?

No. Just that your parents will get on your case and you might do poorly if your school or work schedule requires waking up early.
Yes. This is a bad habit which distorts your normal hormonal rythym. This will mess up thyroid, cortisol and all hormonal processes leading to chronic degenerative disease and premature aging.