Does winter indoor dryness actually make skin itchy or is it a myth?

Yes: Dry skin is more itchy and this has not only to how much fluids you consume but also the humidity in the air.

Some basics: The human skin appears to function best in an environment with 30-40%+ humidity. Heated homes/buildings (especially with radiators) can often dry out the air to below 10% humidity, creating a drying effect on the skin surface & trigger itching in those with sensitive skin. A dry throat is also common. Using a humidifier during the winter will reduce such problems.