Is it concerning for one side of my scrotum to have much larger veins than the other?

Left varicocele. Veins in the scrotum are usually varicocele. Varicocele is more common on the left side. This has to do with the way the vein course back to heart from the testicle. On the left side it takes a hard 90 degree turn to get to the heart. The right side has a more direct route to the heart. The turn on the left can be an obstacle of sorts and make the veins maintain blood closer the testicle.
Probable varicocele. Varicoceles usually occur in the left scrotum and feel like a "bag of worms" if you feel them when you are standing erect. Varicose veins empty and can not be felt when you are lying down. Worth getting a semen analysis and see a urologist, if you plan to have kids. Don't worry if you already have kids. Varicoceles are associated with infertility in some, by no means all, men.