As the others have been saying, it's a slightly bigger picture than just batteries.
What sort of lights are we talking about? Most lights do not easily support lots of different batteries, more due to the physical size than anything.
Anyhow, most LED lights use an electronic driver (circuit board) to control output and modes on a light. This driver will have an operating voltage range. Exceeding this voltage will normally fry them and too low a voltage and they won't likely function, or have very reduced performance/modes.
There are many types of driver depending on what it needs to do.
Most common LEDs require 3.3-3.8v to operate. Although there are some LEDs that require higher voltage and some that require lower.
On this basis, most lights then fall into either using a:
-Buck driver, which lowers the battery voltage down to that needed of the LED. Usually multi battery lights or lots of battery lights.
-Boost driver, which uses a battery below the voltage of the LED and uses more amps to 'boost' the voltage to the correct level.
-Linear driver, which has a battery of about the right voltage for the LED and simply controls current and modes, rather than voltage.
As for batteries.
Li-ion is typically 4.2v when resting and fully charged. However they have a 'nominal' voltage of 3.6 -3.7v, which is why you often see all these figures mentioned. Nominal is the battery under normal load.
When amps are drawn from any battery the voltage of the battery will drop, the higher the draw, the bigger the drop as a rule. This is why different batteries perform differently and to different levels of performance.
Physically larger batteries of the same type will typically outperform smaller versions of the same type, e.g. most 26650 Li-ion will out perform 18650's and 18650's will out perform 14500's.
Same is true with AA's out performing AAA's.
Anyhow, feeding an LED more volts than it needs will result in the need to 'burn' off the excess voltage as heat. But the positive here is, it gives you very good stable output, as the battery voltage will be higher than the LED needs, so less demand on amp draw of the batteries. A 2xCR123a or a 2xLi-ion for example.