If the lights are left unattended in temperature extremes them energizer aa lithium in adapters might be a better idea. A single 18650 cell replaces 3 alkalines.
As TG says, you have to match the voltage.
Yeah, AA energizer lithiums are best for little used lights stored in a vehicle or anywhere there may be extremes in heat and cold.
But if you do get some Li-ion adapters for your Mags you may find yourself using them a lot more as quality Li-ions make these lights much more pleasurable to use.
Most 2D LED lights will run fine on 3.6V Li-ions, but some may not so you need to do some research and there may be some risk as a Li-ion cell will put out about 1 Volt more than a fresh pair of alkalines. If your light is an incandescent, you want to use a 3 cell bulb or a LED drop-in rated for at least 3 cells.
For a 3 cell Mag the Voltage of a single Li-ion is perfect.
For two Li-ions in series you need a bulb or drop-in rated for 6 alkaline cells and you need button top protected Li-ion cells for safety.
I currently only make adapters for single protected Li-ions (with built in mechanical reverse polarity protection) as this is safest for lights without low Voltage cut off, like virtually all C and D cell lights.
If you want more runtime than an 18650, I recommend using a 21700 for C cell lights and 26650 for D cells. I make adapter sets to run 18650's or 21700's in 2 or 3 D cell lights, one adapter is a 18650 or a 21700 to 2 or 3 C cells and the other is a C cell to D cell that allows the smaller 18650 and 21700 cells make contact with the larger D cell light's tail spring without doing any mods, The 18650 and 21700 to C cell adapters work in most C cell lights, no problem as they are large enough to make contact with the smaller C cell sized tail springs.
For D cell lights 26650's are generally big enough to make contact directly with an unmodified tail spring, so I make a single adapters for these.
If you want to make your own adapter, check out Old Lumen's (RIP) vid:
If you want to get into more extensive upgrades for your old Mags, also check out Matt Smith's/Lumencraft vids.