Yup, I hear ya. The cold definitely brings on more crashes.
Shit, two of our VERY experienced track day staff went down at our last NJMP event in October & attributed it mainly to cold tires... but it's not like it poses a significant danger or anything.
They said they'll give you the option to ride... I'd still ride. But I'd ride to the conditions.
Same goes for the rain... crashes are more likely but I'm still out there slip slidin away
Riding in adverse conditions improves your skillset.