windproof materials are probably more important than anything else - especially on the torso and arms which take the bulk of the wind and do the least work compared to the legs.
the key is layering - a good base layer (not necessarily thermal) that wicks sweat away, followed by one or two top layers that are windproof and wicking (I tend to go with a jacket and a gilet). winter long finger gloves that are also windproof and something under the helmet (hat or buff) to keep the bonce warm. a buff around the neck can help as well. and overshoes with either decent winter socks or perhaps even sealskinz which are water and windproof.
other than that - ride quicker....