Such a thing as 8-9mm dia 40-65mm long lithium battery?

I found a bunch of promotional micro maglights (the kind that use 4 hearing aid batteries) stashed away in a junk drawer and thought I'd give it a try at modding a few with different LEDs or something.

Is there such a thing as a small diameter lithium cell I could use to power them?

Or am I better off sticking with hearing aid batteries

Consider the voltage. A LiIon would max out at 4.2V. A stack of hearing aid batteries are 3V pr. cell (typically). How tall is the stack?
My point is just that even if you find one in the right dimensions, it's not certain it would do you any good, sorry.

Hearing aid batteries are usually zinc-air cells, with a nominal voltage of 1.2v. If they are truly zinc-air cells then something like a 10440 might work if it fits into the light.

What is the number on the cells? A CR prefix denotes lithium primary cells, which are 3v. LR and SR prefixes denote alkaline or silver oxide cells, which are only 1.5/1.2 volts. An A or ZA prefix is a true zinc-air cells, with a voltage of 1.2v. I doubt that they are zinc-air cells, since a zinc-air cell dies in a few weeks regardless of whether or not it is being used.

There are such batteries available, they are used in cheap and simple e-cig devices, but they usually are LiPo, not LiIon.