The_LED_Museum
*Retired*
[ QUOTE ]
DaGunn said:
Craig,
Since I submitted both the 8-LED (1xAA) and 21-LED (2xCR123A) lights to you for review and you decided to post links to your reviews of said lights in this forum, I will respond to your less than stellar reviews in this forum.
I had identified these two LED lights as promising candidates for Christmas gifts and/or modding fodder for fellow CPFers and shipped them to you for review. I explained this to you, pior to sending these lights.
However, the testing methodology that you employ appears to be biased, with an inclination and assumption that ALL lights are either tactical lights, dive lights, or both (e.g., can the flashlight survive 10 impact strikes against concrete, be waterproof, and show no damage?). If so, it passes as a flashlight? (Please, give me some examples of waterproof lights that have passed these tests along with their pricing. Show me any Maglite that has passed this test.) The two lights that I submitted were put forth as inexpensive $10-$15 LED lights This is like me asking you to evaluate a Porsche 911 and you applying a grueling 4x4 test stating that since the Porsche can't go through 3ft of mud and pull two tons of poop it is not worthy of the price it demands.
As to the fair application of your 4x4 tests: I did not read anywhere that you had performed these physically damaging tests to your PK surefires, yet you rated them highly with stars immediately..... whereas, you had written that you needed to spend considerable time with a light before you could award them stars?
Dave
PS -
/ubbthreads/images/graemlins/smoker5.gif -- Yep, your tests stink!
[/ QUOTE ]
Good morning Dave,
I'm very sorry to hear that you disprove of my testing methodology. /ubbthreads/images/graemlins/frown.gif
All flashlights I evaluate on my website receive the same treatment, regardless of whether they are Porches or Pacers.
Flashlights can pass or fail these tests. Those which pass them all rate highly; flashlights that fail one or more of the tests rate less highly.
Let me go to my website and snag the testing methodology text. This is a pretty long text, so please beware...BRB...
------------------------------------------------------------------------------------------------------------------------
Examine the light and the packaging it came in. Note any difficulties I had in removing the light or any of its accessories from the package. Note whether or not it came with batteries.
At this time, I've already opened a template and begun constructing the flashlight's web page.
Install the batteries, and attempt to operate the light without referring to any instructions first. This determines how much difficulty a typical consumer might have if they also fail to read the instructions, or if the instructions are disposed of, crushed out on the carpet like a cigerette, fall in the toilet and are then disposed of in the garbage can or by flushing, or become lost.
Loose accessories included in the same package, such as split rings or lanyards, are also installed at this point, and note is taken of any difficulties I had beyond the ordinary. If the unit turns into a pile of loose parts when opened, this is also noted. When possible, the packaging is preserved for future reference.
The steps needed for correct battery installation are recorded for future reference, and for the benefit of website visitors.
Measure the light output while the batteries are still new. This is done by holding or mounting the light so the LEDs are aligned with a precalibrated bar located exactly 12" from the face of the light sensor, and recording the resulting figure(s). A Tektronix J-16 photometer is used to take this measurement. A Wavetek Meterman LM631 light meter has been in use since mid-2002, and it performs the same duties the Tektronix model does.
Take the flashlight's picture. Say cheese!!
Typically, this would include one photo of the flashlight itself, and a second shot of the light in my hand, which is usable as a general reference to its physical size. Sometimes, a third picture is taken of the light's bezel (head) for inclusion on its web page.
Photograph the light's radiation field on a precalibrated test target. Most lights tested need only one picture of this; however if there are any unusual beam characteristics, I'll adjust the camera exposure to a lower and/or higher value and take another picture or pictures so these artifacts can be seen in the photograph. I may also take an additional picture from 5-6 feet away if the flashlight uses a Luxeon Star LED and is touted to "throw" far away. I don't have access to an outdoor "testing range", so I cannot take any outdoor photographs. And if the flashlight uses a "rear-firing" mechanism and produces a square beam, I'll take another photograph showing this square shape, underexposing the photograph if necessary.
Beam photographs on this target are taken at 12", and the markings on the target itself are exactly 1" apart.
Closely examine the sample(s). This examination is to note the overall fit & finish, and to note any features like split ring attachment points, knurling or texturing, note how the switch operates, the material(s) the flashlight is made from, note the LED lamp (to see if there's anything different about it than in other products), examine the reflector and note whether or not it is actually useful, examine the battery contacts, check for the presence or absence of a catalyst pellet, and to examine it for anything else that would be deemed noteworthy.
Unusual characteristics or special features are noted and/or photgraphed at this point for inclusion on the web page.
For flashlights that aren't specifically marked or sold as "fragile" or "not impact-resistant" and do not look delicate, conduct the first series of drop tests. The first test is from chest high onto tight loop carpet (0.1" thick) with no padding over a linoleum floor. Then the unit is dropped from the same height onto bare linoleum, and any breakage is noted. Finally, the unit is thrown or dropped from approximately 7 feet onto bare linoleum; again any breakage is noted.
For "consumer level" testing I will wander around the test area with the lit flashlight in hand and purposely run into doors or corners, being sure the flashlight is knocked to the floor. Lights with belt holsters are mounted, and then I purposefully get jammed in a metal framed doorway so the flashlight takes the brunt of the "accident". If the unit tears away or becomes broken, that is noted on the web page
Appropriate lights are also "carelessly" tossed into a fairly full metal toolbox a number of times. Sometimes I throw them right in, other times I do a "lay up" against the opened lid, tossing the light in from five or six feet away. (Maybe I should install a miniature basketball hoop on my toolbox).
Smaller to medium sized flashlights may also be left on chairs and then "accidentally" sat on.
If it happens to a flashlight in real life, I try to recreate the same type of occurance here.
Next comes the real abuse. I start by holding the light near the bottom of the barrel, and strike the head or bezel assembly against a concrete sidewalk, and note any breakage. I then hold it by the bezel, and strike the barrel against the same target. The force used is "moderate", something a bit more than might be achieved by sombody spinning around in alarm and whapping their light against a rock face, metal doorway, car door, or other hard surface. The test is repeated until each part of the light has been struck five times, or until breakage occurs.
Flashlights marketed specifically as being "extremely tough" (Mag Lites, Tektite, Princeton Tec, etc.) may also be subject to an even more brutal version of this test, where I swing the light like a ball peen hammer and strike the sidewalk. They may also be run over with a 400 pound motorized wheelchair, or intentionally stomped on. Smaller metal lights tend to do well with this particular test.
Now, assuming I haven't destroyed the test sample (and very few actually are totally ruined by this), I turn the light on and immerse it in a tank of water for a period of time. This can be anywhere from 20 minutes to several days, depending on manufacturer's claims of water resistance. Lights that are *clearly not* water resistant generally aren't dunked, unless I am specifically asked to do that test.
When possible, lights are turned on and off while still submerged.
Now I wait until darkness falls, and then try to use the flashlight for as many situations as I can get myself into. Such situations can include:
Raiding the refrigerator; making a simple snack (sandwich, etc.) using only the flashlight.
Reading a paper or reading a magazine.
Wandering around the house without stubbing toes or breaking lamps.
Unclogging a sink or toliet (or pretending to do that) at 4am with only the test light.
Looking for various objects (TV remote, other flashlights, cigarettes, etc.)
From bed: Groping for the flashlight on the nightstand or floor and turning it on as quickly as possible. Note how easy or hard this is to do in total darkness.
Attempting to change the unit's batteries in total darkness.
Checking the breaker panel.
Some of the brightest lights are also tested by being mounted to the steering arm of my wheelchair and then going out at night to see how well they perform outdoors as a headlight. As of 10-15-04, a SureFire L6 is being used in this manner.
Finally, many of the units will be "adopted" as a daily carry light, and then used in any situation calling for the use of a flashlight. This can go on for several months, and any breakage or malfunction that crops up would be noted in the "Updates" section of that light's web page.
Lights which are "loaners" (sent by private individuals / fans of the website) are not subject to some of these tests, particularly those which could result in breakage. Most people who loan their lights for testing on this website want them returned in working order. These lights are shown in the pick list with (No longer have) in red lettering. Please do not e-mail me asking me to compare one of those to some other light - there is no way for me to re-test or compare what I don't have anymore. These are also marked with a red "X" in a yellow box when they're moved to their right place in the left-frame menu.
When time and instrument availability permits, run-time tests are performed at this point. This determines how long the batteries last if the unit were used in a full-time situation, such as the admittedly extreme case of nuclear winter or the slightly more likely scenario of being trapped someplace dark after a natural disaster.
All lights which aren't "loaners" are kept available for both future testing and to use for comparisons with other lights when somebody e-mails me asking to perform such a comparison. No test units are sold or given away, as doing so would eliminate the possibility of doing future comparisons with that sample. When a product is received for evaluation, it will not normally be returned unless found to be defective. This is to ensure the sample remains available for long-term testing or comparisons as needed.
------------------------------------------------------------------------------------------------------------------------
DaGunn said:
Craig,
Since I submitted both the 8-LED (1xAA) and 21-LED (2xCR123A) lights to you for review and you decided to post links to your reviews of said lights in this forum, I will respond to your less than stellar reviews in this forum.
I had identified these two LED lights as promising candidates for Christmas gifts and/or modding fodder for fellow CPFers and shipped them to you for review. I explained this to you, pior to sending these lights.
However, the testing methodology that you employ appears to be biased, with an inclination and assumption that ALL lights are either tactical lights, dive lights, or both (e.g., can the flashlight survive 10 impact strikes against concrete, be waterproof, and show no damage?). If so, it passes as a flashlight? (Please, give me some examples of waterproof lights that have passed these tests along with their pricing. Show me any Maglite that has passed this test.) The two lights that I submitted were put forth as inexpensive $10-$15 LED lights This is like me asking you to evaluate a Porsche 911 and you applying a grueling 4x4 test stating that since the Porsche can't go through 3ft of mud and pull two tons of poop it is not worthy of the price it demands.
As to the fair application of your 4x4 tests: I did not read anywhere that you had performed these physically damaging tests to your PK surefires, yet you rated them highly with stars immediately..... whereas, you had written that you needed to spend considerable time with a light before you could award them stars?
Dave
PS -
/ubbthreads/images/graemlins/smoker5.gif -- Yep, your tests stink!
[/ QUOTE ]
Good morning Dave,
I'm very sorry to hear that you disprove of my testing methodology. /ubbthreads/images/graemlins/frown.gif
All flashlights I evaluate on my website receive the same treatment, regardless of whether they are Porches or Pacers.
Flashlights can pass or fail these tests. Those which pass them all rate highly; flashlights that fail one or more of the tests rate less highly.
Let me go to my website and snag the testing methodology text. This is a pretty long text, so please beware...BRB...
------------------------------------------------------------------------------------------------------------------------
Examine the light and the packaging it came in. Note any difficulties I had in removing the light or any of its accessories from the package. Note whether or not it came with batteries.
At this time, I've already opened a template and begun constructing the flashlight's web page.
Install the batteries, and attempt to operate the light without referring to any instructions first. This determines how much difficulty a typical consumer might have if they also fail to read the instructions, or if the instructions are disposed of, crushed out on the carpet like a cigerette, fall in the toilet and are then disposed of in the garbage can or by flushing, or become lost.
Loose accessories included in the same package, such as split rings or lanyards, are also installed at this point, and note is taken of any difficulties I had beyond the ordinary. If the unit turns into a pile of loose parts when opened, this is also noted. When possible, the packaging is preserved for future reference.
The steps needed for correct battery installation are recorded for future reference, and for the benefit of website visitors.
Measure the light output while the batteries are still new. This is done by holding or mounting the light so the LEDs are aligned with a precalibrated bar located exactly 12" from the face of the light sensor, and recording the resulting figure(s). A Tektronix J-16 photometer is used to take this measurement. A Wavetek Meterman LM631 light meter has been in use since mid-2002, and it performs the same duties the Tektronix model does.
Take the flashlight's picture. Say cheese!!
Typically, this would include one photo of the flashlight itself, and a second shot of the light in my hand, which is usable as a general reference to its physical size. Sometimes, a third picture is taken of the light's bezel (head) for inclusion on its web page.
Photograph the light's radiation field on a precalibrated test target. Most lights tested need only one picture of this; however if there are any unusual beam characteristics, I'll adjust the camera exposure to a lower and/or higher value and take another picture or pictures so these artifacts can be seen in the photograph. I may also take an additional picture from 5-6 feet away if the flashlight uses a Luxeon Star LED and is touted to "throw" far away. I don't have access to an outdoor "testing range", so I cannot take any outdoor photographs. And if the flashlight uses a "rear-firing" mechanism and produces a square beam, I'll take another photograph showing this square shape, underexposing the photograph if necessary.
Beam photographs on this target are taken at 12", and the markings on the target itself are exactly 1" apart.
Closely examine the sample(s). This examination is to note the overall fit & finish, and to note any features like split ring attachment points, knurling or texturing, note how the switch operates, the material(s) the flashlight is made from, note the LED lamp (to see if there's anything different about it than in other products), examine the reflector and note whether or not it is actually useful, examine the battery contacts, check for the presence or absence of a catalyst pellet, and to examine it for anything else that would be deemed noteworthy.
Unusual characteristics or special features are noted and/or photgraphed at this point for inclusion on the web page.
For flashlights that aren't specifically marked or sold as "fragile" or "not impact-resistant" and do not look delicate, conduct the first series of drop tests. The first test is from chest high onto tight loop carpet (0.1" thick) with no padding over a linoleum floor. Then the unit is dropped from the same height onto bare linoleum, and any breakage is noted. Finally, the unit is thrown or dropped from approximately 7 feet onto bare linoleum; again any breakage is noted.
For "consumer level" testing I will wander around the test area with the lit flashlight in hand and purposely run into doors or corners, being sure the flashlight is knocked to the floor. Lights with belt holsters are mounted, and then I purposefully get jammed in a metal framed doorway so the flashlight takes the brunt of the "accident". If the unit tears away or becomes broken, that is noted on the web page
Appropriate lights are also "carelessly" tossed into a fairly full metal toolbox a number of times. Sometimes I throw them right in, other times I do a "lay up" against the opened lid, tossing the light in from five or six feet away. (Maybe I should install a miniature basketball hoop on my toolbox).
Smaller to medium sized flashlights may also be left on chairs and then "accidentally" sat on.
If it happens to a flashlight in real life, I try to recreate the same type of occurance here.
Next comes the real abuse. I start by holding the light near the bottom of the barrel, and strike the head or bezel assembly against a concrete sidewalk, and note any breakage. I then hold it by the bezel, and strike the barrel against the same target. The force used is "moderate", something a bit more than might be achieved by sombody spinning around in alarm and whapping their light against a rock face, metal doorway, car door, or other hard surface. The test is repeated until each part of the light has been struck five times, or until breakage occurs.
Flashlights marketed specifically as being "extremely tough" (Mag Lites, Tektite, Princeton Tec, etc.) may also be subject to an even more brutal version of this test, where I swing the light like a ball peen hammer and strike the sidewalk. They may also be run over with a 400 pound motorized wheelchair, or intentionally stomped on. Smaller metal lights tend to do well with this particular test.
Now, assuming I haven't destroyed the test sample (and very few actually are totally ruined by this), I turn the light on and immerse it in a tank of water for a period of time. This can be anywhere from 20 minutes to several days, depending on manufacturer's claims of water resistance. Lights that are *clearly not* water resistant generally aren't dunked, unless I am specifically asked to do that test.
When possible, lights are turned on and off while still submerged.
Now I wait until darkness falls, and then try to use the flashlight for as many situations as I can get myself into. Such situations can include:
Raiding the refrigerator; making a simple snack (sandwich, etc.) using only the flashlight.
Reading a paper or reading a magazine.
Wandering around the house without stubbing toes or breaking lamps.
Unclogging a sink or toliet (or pretending to do that) at 4am with only the test light.
Looking for various objects (TV remote, other flashlights, cigarettes, etc.)
From bed: Groping for the flashlight on the nightstand or floor and turning it on as quickly as possible. Note how easy or hard this is to do in total darkness.
Attempting to change the unit's batteries in total darkness.
Checking the breaker panel.
Some of the brightest lights are also tested by being mounted to the steering arm of my wheelchair and then going out at night to see how well they perform outdoors as a headlight. As of 10-15-04, a SureFire L6 is being used in this manner.
Finally, many of the units will be "adopted" as a daily carry light, and then used in any situation calling for the use of a flashlight. This can go on for several months, and any breakage or malfunction that crops up would be noted in the "Updates" section of that light's web page.
Lights which are "loaners" (sent by private individuals / fans of the website) are not subject to some of these tests, particularly those which could result in breakage. Most people who loan their lights for testing on this website want them returned in working order. These lights are shown in the pick list with (No longer have) in red lettering. Please do not e-mail me asking me to compare one of those to some other light - there is no way for me to re-test or compare what I don't have anymore. These are also marked with a red "X" in a yellow box when they're moved to their right place in the left-frame menu.
When time and instrument availability permits, run-time tests are performed at this point. This determines how long the batteries last if the unit were used in a full-time situation, such as the admittedly extreme case of nuclear winter or the slightly more likely scenario of being trapped someplace dark after a natural disaster.
All lights which aren't "loaners" are kept available for both future testing and to use for comparisons with other lights when somebody e-mails me asking to perform such a comparison. No test units are sold or given away, as doing so would eliminate the possibility of doing future comparisons with that sample. When a product is received for evaluation, it will not normally be returned unless found to be defective. This is to ensure the sample remains available for long-term testing or comparisons as needed.
------------------------------------------------------------------------------------------------------------------------
