If you want a summary, my opinion (supported by data) is that it will definitely prolong the life (and slow self-discharge) of pretty much ANY battery type when you store it cool. But putting it in the fridge makes more sense than the freezer. Fewer opportunities for problems to occur in less extreme environments, and you still get most of the benefits. Either way, make sure you store them individually, in a watertight container. Any small gain you get from storing them cool will be more than offset by problems if you fail to let them warm up in the container (to avoid condensation) before use.
Personally, I don't think it is worth the effort. I like my cells to be ready to go. Maybe if you have a super expensive battery pack, and you know you won't be using it for a long period, it makes sense to store it in the fridge. But for typical batteries, I just don't see the value. But that part is definitely up to the consumer...your priorities may differ.
If you haven't had any problems, I wouldn't worry about it. The above was what I would consider best practices, but not absolutely required. Still, I would think about making some changes in the future. Lithium-ion cells do demand respect and proper care.