Oh, I think I see now. You seem to be thinking about total energy, not mAh capacity. 3.7V cells do contain more energy per mAh than 1.5V cells.
The way to calculate energy is to multiply the voltage and the mAh capacity together. That gives you a figure in mWh (milliwatt-hours).
A couple of examples to compare:
Panasonic Eneloop Pro AA NiMH
1.2V x 2500mAh = 3000mWh or 3Wh.
Samsung INR18650-25R 18650 Li-Ion
3.6V x 2500mAh = 9000mWh or 9Wh.
Both of those cells have the same capacity in mAh, but the Li-Ion cell contains three times the energy, and it's the energy that determines how long your light or powerbank will run for.
Nine times out of ten, you don't need to worry about energy, though, because you'll normally be comparing like with like. For example, you'll only ever be comparing different 18650 cells for your 18650 battery pack. That means you can use the mAh capacity as a convenient shorthand for total energy, because all the cells you're comparing will have the same voltage anyway.