Disclaimer: The following information is several years old and may not apply to the chemistry in newer batteries.
Having said that, I have intentionally taken Alkaline, NiMH, and several types of Lithium batteries out winter camping and left them sitting, exposed, in weather that has been as cold as -7 Fahrenheit.
Below about 20 degrees (F) the Alkaline batteries showed a significant drop in performance and at somewhere around 2 degrees (F) they were essentially useless, I believe that they had actually frozen at that point. Interestingly with the NiMH batteries I was using, their output actually seemed to -improve- just slightly from roughly 35 degrees (F) down to about 25 degrees. Once they got below about 5 degrees then things seemed to taper off pretty fast.
The lithium chemistry batteries...just kept going.
Because of that testing, all of my automobile flashlights/led "flares" and so forth have lithium batteries in them and so do the outdoor sensors for the wife's "weather station".