Lumens tells you the total amount of light. A 100 watt light bulb produces a lot of light (1400 to 1600 lumens) but the light goes in all directions, half of it wasted going behind you.
A CPF member once started a thread titled something like "My SMJLED flashlight (~15 lumens) is brighter than my 100W lightbulb (~1500 lumens)" That is because the flashlight has a reflector that produces a focussed beam that goes out farther in one direction than the unfocused light of the lightbulb.
The measurement of how bright the center of the beam is is done 2 ways Lux or candlepower (cp). CPF members usually use Lux.
So you need 2 numbers:
Lumens for total light output;
Lux for how bright the hotspot is.
-----
The big blob of dim light on the left is from an MTE '900 lumens' P7 (probably 400 lumens actual).
The much brighter dot on the right is from a Terralux '50 lumens' TLE-1F bulb (probably 30 lumens actual).
So lumens does not tell you how much throw (beam distance) a light produces.
I got the TLE-1F bulb to produce such a tight beam by using a huge 4 inch reflector.
-----
There are a few ways in which lumen numbers are produced.
Small relatively poor Chinese companies simply can not afford to pay for a calibrated integrating sphere (do you know how much it costs to fly an expert over to calibrate the thing every few months? - in US $ too :sick2
So they grab a number off the LED manufacturer's spec sheet - 900 lumens for the P7, 700 lumens for the MCE, 320 lumens for the XPG, 220-260 lumens for the XRE (depends on bin). So you know you are seeing make believe when you see those numbers.
Better manufacturers calculate or measure the current to the LED, then look up what the LED should produce. This gives LED lumens. this allows us to estimate OTF (out the front) lumens. Light transmission range from 65% for a Maglite to 81% for a Fenix.
The best manufacturers actually have their lights measured either in-house or by an outside lab. Surefire, 4sevens, Arc, all the US big boys - Mag, Streamlight, UK, Pelican, Princeton Tec.
-
But is a single number that useful in telling you what you can expect in normal use?
4sevens Quark Mini AA2 is rated 180 OTF lumens on high.
Tri-force got these measurements:
Energizer L91 (E2)
187.9 1 sec
147.9 30 sec
145.5 1 min
144.2 2 min
143.0 3 min
Duracell AA alkaline
162.4 1 sec
151.5 30 sec
147.9 1 min
145.5 2 min
140.6 3 min
Yes the initial lumens is above spec with Energizer L91 (E2) lithium primaries but in normal operation after the first few minutes you are running around 140 lumens.
So use the lumen numbers, even the good ones as a guide, not as an absolute.