Hello Albinoni,
Lumens is a measure of the light output of the light. In general, more power is needed to generate more lumens. There are advances that allow more lumens from the same previous power, and efficiencies of circuits and contact resistance also need to be factored in.
Once the light is produced, you have to decide what to do with it. If you are looking for an area light, you shape your light into a wide beam. If you are trying to see something at a distance, you shape the light to focus the beam and it becomes directional.
Shaping the light involves many factors including the use of lenses and reflectors. You can shape the light so you only have a hot spot, and no spill. This will project the light further, but you end up with a bit of tunnel vision. You can shape the light with a pin point hot spot and some spill, or a large hot spot and a lot of spill, and so on. Your beam shape is dependent on your usage. For example, bicycle riding at night involves being able to see obstacles or hazards in the distance, but you also need side spill as well.
Light falls off or dims with the square of distance. To determine how far a beam will travel, you measure lux at 1 meter, or foot candles at 1 foot, take the square root of that measurement, and you have the distance in meters (for lux) or feet (for foot candles) that the light will dim down to 1 lux or 1 foot candle.
The more light output you have (lumens) the more options you have to shape the beam into something that works for you.
Lumens are measured by introducing the beam into an integration sphere and measuring the light using a light meter that has been calibrated for the spectrum of the light you are measuring. The integration sphere measurement set up runs from $6000 - $20000 and up. In addition, the calibration sources for the light meter used with the sphere are very expensive and they are considered consumables.
Because of the high costs, most comments about lumens are a guess. Some guesses are closer than others, but in the end they are still guesses. Some people have built a home made integration sphere or box, but the weak link now falls back to the light meter used for the measurements.
I am sorry to say that lux measurements are not much better. Most lux meters are calibrated for an incandescent light source. If you are measuring incandescent light, you are probably pretty close, as long as you pay attention to alignment and have accurate distances to work from. LED's emit light in a different part of the spectrum. A light meter that is calibrated for incandescent light may not give accurate readings when measuring LED light.
If you really MUST know the output of a light, I would suggest sending it to a lighting test laboratory. They can run a series of tests on your light and give you lumens, lux, and the spectrum of the light. The cost is around $200/light, but sometimes they give bulk discounts.
Lamp and LED manufactures have integration sphere set ups and they check the lamps and LED's and report lumens at a specific, or a series of currents. These numbers are usually very good, but they are just for the bare lamp or emitter. Put that into a flashlight with a reflector or focusing lens, add a protection lens and your initial lumens are reduced.
The incandescent people have found that lamp lumens are reduced by 35 - 40% by the time they come out of the "torch." They refer to torch lumens and lamp lumens. If you are looking at a lamp you want to use and are wondering how much light you can expect to get from it in a flashlight, you take the lamp lumens and multiply it by 0.6 or 0.65 to get an estimate of torch lumens.
Some manufacturers give torch lumens. SureFire is an example. However, unless the light is regulated, the lumen value is not static. It starts out high and drops down as the batteries become depleted. I believe SureFire measures their lights with batteries and picks a point part way through the run time to state their lumen value. Another way to approach this is to drive the light at a voltage (picked by the light manufacturer) and report the lumen value. This is only accurate when your battery voltage is the same as the voltage used for the test. Some manufacturers insist that lumen measurements be reported within 5 seconds of the light being turned on.
As you can see, measuring light is a bit of a mess. If you have an interest in this, pick up a light meter and get a feel for it. Use it with a wide variety of lights and remember that your results may not totally correlate with mine. We should be in the same neighborhood, but who knows...
In the end, lumen measurements should be something that we can readily use, but inconsistencies in measurement methods have diluted its usefulness. However, it does provide a good gross indicator. A focused flashlight that has been measured at 150 lumens will be brighter than one measured at 65 lumens.
Tom