Li-ion rechargeable batteries have a wide voltage range during use. Fully charged, most of them measure 4.2 volts. Fully drained, perhaps as low as 2.5 volts or 2.8 volts, although most of the energy is gone by the time you reach 3.0 volts. So the typical operating range is, say, 3.0v to 4.2v.
This puts battery makers in a quandary. How many volts should they say their batteries have? The answer they chose is use the "nominal" voltage, 3.6 or 3.7 volts, which is like an average. That is the number printed on the label. Some makers use 3.6 volts; others, 3.7.
This can be confusing, because there is another kind of lithium battery, the LiFePO2, that has a voltage range between 2.5 and 3.6 volts. Fully charged, they have 3.6 volts. Their nominal voltage, however, is only 3.2 volts. This latter number is the one printed on the label.
Yet a third kind of Li-ion battery exists. It is similar to the 4.2-volt batteries, except it can be charged higher. It is fully charged at 4.35 volts. Its nominal voltage is usually given as 3.7 or 3.8 volts.
There is also a confusion as to exactly what voltage an RCR123 battery has. Some companies use that designation for 3.7-volt batteries that can be charged up to 4.2 volts. Other companies reserve that name for 3.2-volt LiFePO2 batteries that can be charged to a maximum of 3.6 volts. See the article
CR123A and rechargeable substitutes by HKJ for more information about this.
The bottom line is that you have to know what kind of battery your device will accept, and learn how to properly charge it.