Re: Some thing new at cpfreviews.com -- Simple Comparison Engine... DONE!
Your review process is far more complex than mine.
To wit:
================================================================
Testing of flashlights for The Punishment Zone consists of a number of discrete steps, each designed to place the sample in an abusive environment, to measure it using known physical parameters, and to determine how usable the unit might be once it ends up in the hands of a real consumer.
These steps are:
Examine the light and the packaging it came in. Note any difficulties I had in removing the light or any of its accessories from the package. Note whether or not it came with batteries.
At this time, I've already opened a template and begun constructing the flashlight's web page.
Install the batteries, and attempt to operate the light without referring to any instructions first. This determines how much difficulty a typical consumer might have if they also fail to read the instructions, or if the instructions are disposed of, crushed out on the carpet like a cigerette, fall in the toliet and are then disposed of in the garbage can or by flushing, or become lost.
Loose accessories included in the same package, such as split rings or lanyards, are also installed at this point, and note is taken of any difficulties I had beyond the ordinary. If the unit turns into a pile of loose parts when opened, this is also noted. When possible, the packaging is preserved for future reference.
The steps needed for correct battery installation are recorded for future reference, and for the benefit of website visitors.
Measure the light output while the batteries are still new. This is done by holding or mounting the light so the LEDs are aligned with a precalibrated bar located exactly 12" from the face of the light sensor, and recording the resulting figure(s). A Tektronix J-16 photometer is used to take this measurement. A Wavetek Meterman LM631 light meter has been in use since mid-2002, and it performs the same duties the Tektronix model does.
Take the flashlight's picture. Say cheese!!
Typically, this would include one photo of the flashlight itself, and a second shot of the light in my hand, which is usable as a general reference to its physical size. Sometimes, a third picture is taken of the light's bezel (head) for inclusion on its web page.
Photograph the light's radiation field on a precalibrated test target. Most lights tested need only one picture of this; however if there are any unusual beam characteristics, I'll adjust the camera exposure to a lower and/or higher value and take another picture or pictures so these artifacts can be seen in the photograph. I may also take an additional picture from ~10 feet away if the flashlight uses a Luxeon Star LED and is touted to "throw" far away. I don't have access to an outdoor "testing range", so I cannot take any outdoor photographs. And if the flashlight uses a "rear-firing" mechanism and produces a square beam, I'll take another photograph showing this square shape, underexposing the photograph if necessary.
Beam photographs on this target are taken at 12", and the markings on the target itself are exactly 1" apart.
Take a spectrographic analysis, using the Ocean Optics USB2000 Spectrometer on loan from
WWW.TWO-CUBED.COM.
Mount the light to a carrier and fire up the ProMetric beam profile analyser and start taking readings. This analysis will display all aspects of the light's beam, highlighting various defects, rings, and other artifacts. This instrument is also capable of taking more standard photometric measurements as well, which will be posted with the ProMetric's chart on each light's page. This will not eliminate the need to take the measurement with the Tektronix photometer; but is intended to augment it. The ProMetric will also not replace the traditional beam photograph, as it records its images in black & white and/or false color; while the traditional picture will be used to show how you might see the beam with your own eyes.
A seperate computer system will be running specifically for the ProMetric and for a recording meter that gets used later on in testing.
Closely examine the sample(s). This examination is to note the overall fit & finish, and to note any features like split ring attachment points, knurling or texturing, note how the switch operates, the material(s) the flashlight is made from, note the LED lamp (to see if there's anything different about it than in other products), examine the reflector and note whether or not it is actually useful, examine the battery contacts, check for the presence or absence of a catalyst pellet, and to examine it for anything else that would be deemed noteworthy.
Unusual characteristics or special features are noted and/or photographed at this point for inclusion on the web page.
For flashlights that aren't specifically marked or sold as "fragile" or "not impact-resistant" and do not look delicate, conduct the first series of drop tests. The first test is from chest high onto medium loop carpet over an unknown flooring material. Then the unit is dropped from the same height onto concrete, and any breakage is noted.
For "consumer level" testing I will wander around the test area with the lit flashlight in hand and purposefully run into doors or corners, being sure the flashlight is knocked to the floor. Lights with belt holsters are mounted, and then I purposefully get jammed in a metal framed doorway so the flashlight takes the brunt of the "accident". If the unit tears away or becomes broken, that is noted on the web page
Smaller to medium sized flashlights may also be left on chairs and then "accidentally" sat on.
If it happens to a flashlight in real life, I try to recreate the same type of occurance here.
Next comes the real abuse. I start by holding the light near the bottom of the barrel, and strike the head or bezel assembly against a concrete patio floor, and note any breakage. I then hold it by the bezel, and strike the side of the tailcap against the same target. The force used is "moderate", something a bit more than might be achieved by sombody spinning around in alarm and whapping their light against a rock face, metal doorway, car door, or other hard surface. The test is repeated until each part of the light has been struck five times, or until breakage occurs.
Flashlights marketed specifically as being "extremely tough" (Mag Lites, Tektite, Princeton Tec, etc.) may also be subject to an even more brutal version of this test, where I swing the light like a ball peen hammer and strike the concrete patio floor. They may also be run over with a 450 pound motorised wheelchair, or intentionally stomped on. Smaller metal lights tend to do well with this particular test.
Now, assuming I haven't destroyed the test sample (and very few actually are totally ruined by this), I turn the light on and immerse it in a tank of water (a sink or a cistern is typically used for this test) for a period of time. This can be anywhere from 1 minute to several days, depending on manufacturer's claims of water resistance. Lights that are clearly *not* water resistant generally aren't dunked, unless I am specifically asked to do that test.
When possible, lights are turned on and off while still submerged.
Flashlights or other products which were destroyed during or after testing will receive a (
) icon next to their listing on this website. This effectively acts like an (
) icon; in that I will no longer be able to perform additional testing or comparative analyses on the product.
Now I wait until darkness falls, and then try to use the flashlight for as many situations as I can get myself into. Such situations can include:
Raiding the refrigerator; making a simple snack (sandwich, etc.) using only the flashlight.
Reading a paper or reading a magazine.
Wandering around the house without stubbing toes, breaking lamps, or stepping on squeaky little rat tails.
Unclogging a sink or toliet (or pretending to do that) at 4am with only the test light.
Looking for various objects (TV remote, other flashlights, cigerettes, etc.)
From bed: Groping for the flashlight on the nightstand or floor and turning it on as quickly as possible. Note how easy or hard this is to do in total darkness.
Attempting to change the unit's batteries in total darkness.
Checking the breaker panel.
Some of the brightest lights are also tested by being mounted to the steering arm or front kick panel of my electric wheelchair and then going out at night to see how well they perform outdoors as a headlight.
Finally, many of the units will be "adopted" as a daily carry light, and then used in any situation calling for the use of a flashlight. This can go on for several months, and any breakage or malfunction that crops up would be noted in the "Updates" section of that light's web page.
Products (flashlights, lasers, etc.) which are "loaners" (sent by private individuals / fans of the website) are not subject to some of these tests, particularly those which could result in breakage. Most people who loan their lights for testing on this website want them returned in working order. These lights are shown in the pick list with (No longer have) in red lettering. Please do not e-mail me asking me to compare one of those to some other light - there is no way for me to re-test or compare what I don't have anymore. These are also marked with a red "X" in a yellow box when they're moved to their right place in the left-frame menu.
When time and instrument availability permits, run-time tests are performed at this point. This determines how long the batteries last if the unit were used in a full-time situation, such as the admittedly extreme case of nuclear winter or the slightly more likely scenario of being trapped someplace dark after a natural disaster.
================================================================
