Yes, I am (as I already said) making an assumption about density, namely that there isn't a compound that has both a high energy density and a high mass density. Lithium chemistries are all lightweight and have the highest energy densities. Of course it's possible there will be some small difference between certain lithium chemistries where you can choose between the smallest volume or the lowest weight for the same energy density, but AFAIK there isn't enough in it to be worth caring about.
Of course, whether we choose weight or volume is largely irrelevant as long as we pick one or do both. That's manageable. What's unmanageable and full of personal opinions that will never agree is "size", meaning length or diameter or some combination of these that isn't the volume. Given 16340 and 14500 both have max capacities of 750mAh, it's down to personal opinion whether you prefer shorter and fatter or longer and thinner. They both have the same energy capacity and approximately the same weight and volume too. What is perhaps interesting is that the 14500 looks a lot bigger, which is because volume is directly proportional to the length of a cylinder, but proportional to the square of the diameter. Looks can be deceiving though.
I did just weigh a 14500 and 16340 and found the 16340 to be marginally lighter (and therefore of slightly lesser volume), although they were different makes so that's maybe not a 100% fair test. Plus it has a sample size of 1. I would argue that these cells are essentially indistinguishable in either volume, weight or capacity, but are considerably different in terms of length and diameter.
Since nobody else seems interested in doing it, I weighed a few cells. Here are the results:
16340: 750 mAh @ 19.0g = 39.5 mAh/g
18650: 3400 mAh @ 47.4g = 71.7 mAh/g
CR123A: 1300 mAh @ 16.4g = 79.8 mAh/g
Therefore, perhaps surprisingly, the winner is the CR123A primary with the 18650 as runner up. I'm a bit surprised as I thought the 18650 would dominate. There are also 1500mAh CR123As but I don't know if those would be heavier - it's possible those have an even higher energy density, although it's also possible the capacity is overstated. Of course, primaries have a lot of disadvantages, as already noted. What is striking about these numbers is that, despite being the same chemistry, an 18650 has almost double the energy density of a 16340 or 14500.
For completeness, I would be interested in the figures for a 26650 for comparison, but I don't have one to weigh. I did try googling for specs, but I don't think manufacturers generally bother publishing the weight of their cells.
Of course there is one further practical consideration which is that once you have your chosen cell, you need a light to put it in. One thing I have noticed is that while there are quite a few small AA/14500 and CR123/16340 sized lights, most 18650 lights are a lot bigger than they need to be. This means you may find much bigger differences in overall system efficiency when you start looking at not just the cell, but the entire light (the cell, the body, the circuitry, the LED). Also, while smaller lights may be more size-efficient in terms of design and construction, they may be less efficient overall because of the relatively fixed overheads (size of reflector, switch, circuitry, etc.)
For comparison, I weighed a couple of lights as well (batteries installed). Here are those results:
Nitecore P12 + 3400 mAh 18650 @ 137.4g = 24.8 mAh/g
EagleTac D25C Clicky Ti + 750 mAh 16340 @ 57.1g = 13.14 mAh/g
EagleTac D25A Clicky Ti + 750 mAh 14500 @ 48.6g = 15.4 mAh/g
Of course once we are talking about whole lights, weight isn't really a good proxy for volume anymore, but it's still a simple-yet-useful guide to overall size-efficiency. It would be even more useful if we replaced the mAh rating of the battery with the lumen-hours rating of the light, but that throws up even more issues about how to fairly measure lumen-hours across different output levels, different types of regulation, PWM vs direct drive, and so on.