Consumer Reports Assessment of Automotive Headlights

In the July 2015 issue of CR, the staff has this to say about headlights:

Regarding HID: "In our tests, HID headlights generally haven't performed any better than basic halogen lights overall."

Regarding LED: "………a technology that shows little benefit in our tests."

Regarding the LED headlights in the 2015 Cadillac Escalade: "….has the best performing headlights we've ever tested."

Yeah, they also rave (justifiably) about the LED headlamps on the Corolla, which contradicts their dismissal of LED as "a technology that shows little benefit in [their] tests".

Consumer Reports' philosophical preferences in headlamp performance are flawed, their headlamp testing protocol is flawed, the conclusions they draw from it are flawed, and their test results are out of accordance with the research and testing done just about everywhere else. For example, take a look at this very recent research by AAA (infographic here, fact sheet with seeing distance breakdown by headlamp type here). In general the results are probably solid, certainly a good bit more reliable than Consumer Reports'. I would like to see the raw data behind the AAA study though I probably won't be able to, but their findings pass a basic reality test and do comport with other tests done around the world.

(The significance of their results with headlamp lens restoration products, while probably true as far as they actually go, are overhyped in AAA's summary and in general-news media pickups of this news release.)

Thanks for the links Virgil.

I'm looking at a new Toyota Avalon this fall and led headlights are one of my desires. I hope they are good because a lot of my driving is on rural unlit roads.

Toyota's phasing in really good LED headlamps. Buy without hesitation.

-Virgil- said:

………….Consumer Reports' philosophical preferences in headlamp performance are flawed, their headlamp testing protocol is flawed, the conclusions they draw from it are flawed, and their test results are out of accordance with the research and testing done just about everywhere else………..

Click to expand...

Here is how CR does their test:

Direct quote removed, please see CPF Rule 5......Bill

http://www.consumerreports.org/cro/news/2012/07/how-consumer-reports-tests-car-headlights/index.htm

The last sentence reveals how proud they are of their test protocol. What stands out to me is no mention of their methodology being guided by national or international technical standards for performing these tests.

Their test protocol was designed by a company called Guide Light. Guide had been GM's in-house lighting division for many years, which meant they mostly made cheap lighting; their main innovations involved making legal lights cheaper and cheaper. As far as I'm aware, there was only one Guide engineer worth a tinker's damn. I don't know him personally, but Daniel Stern speaks highly of him, he "landed well" after Guide died, and his recent work is very impressive. I can only imagine it must have been totally frustrating for a good engineer to work at Guide.

When GM spun off Guide, it sank like a lead balloon. Nobody wanted to buy their outmoded lamps. Literally nobody; Guide never sold a single lamp design to any automaker other than GM, and even GM ran away when they discovered the world's much less backwards lighting suppliers.

That's who Consumers Union hired to design a headlamp test protocol in the early 2000s (just a few years before Guide finished dying). Naturally, to Guide, a good headlamp was a Guide headlamp, so the CR headlamp test prefers low beams with no cutoff and sees no value in newfangled high tech like HID and LED.

Thanks for that interesting info about Guide Light.
I am a CR subscriber, and often wonder "what are they thinking?" For example, most of their performance testing of winter tires is done under warm temperature conditions.

If you think my opinion of CR is unfavorable, you should hear/see Stern about it!

-Virgil- said:

If you think my opinion of CR is unfavorable, you should hear/see Stern about it!

Click to expand...

Look at the product "reviews" that CPF members post ... are you really surprised?


This is what the ignorant do when they create "tests" ..... real touchy feely. However, proper photometric testing would be far more useful, repeatable, accurate, unbiased ....

CU seems to have a LOT of flawed test designs.

You'd think a company who's raison d'etre is product testing, would do a better job of it.


Some doozies stick in my mind...

They compared HEPA filtered vacuum cleaner's exhaust cleanliness to a conventional vacuum cleaner's exhaust, and concluded that a HEPA filter did not reduce the particles in the emissions.

They used wood flour as the "challenge particulate"...which are ~ 200 micron particles.

A HEPA filter can take out particles closer to 0.3 microns...thousands of times smaller than wood flour.


An ordinary vacuum cleaner filter might take out particles as small as 100 microns, about the size of pollen for example. A mold spore might be closer to 2 microns, etc.


So, by using a 200 micron challenge particle, its like comparing a chain link fence to a screen....using bowling balls as the challenge, and seeing that the chain link fence stopped just as many bowling balls as the screen, and concluding that screens must not filter any better than the fencing.

Even using ping pong balls instead of bowling balls would have shown a difference, but, no, they used bowling balls.




The other was bolting a giant 1,500 lb outrigger onto a wee Suzuki Samari, to catch it in case it tipped over in an emergency maneuver...and then not realizing that the giant 1,500 lb outrigger was making it tip over.



It goes on and on...luckily for them, John Q Public is so generally clueless, that their vouching for, or condemnation of products is blithely accepted as valid.



So HEPA filters don't filter better than regular filters, halogen headlights are better than HID or LED, etc.

If you read the AAA docs, you will see repeated references to cleaning the headlights/restoring the headlights, and so forth.

They DON'T mention that they are pushing their newly refurbished AAA Car Care Centers, and, that if you stop in, you will see giant displays telling you to have their professionals clean and polish your headlights.

Teej, I didn't know that's how they "tested" vacuum filtration, but I believe it. Some years ago they "tested" motor oil filters and their recommendation was to buy any brand, it didn't matter, they're all about the same. The "test" they used to determine this was, as I remember, seemingly random and completely disconnected from reality in a very similar manner to the wood flour "test" you describe. And CR seemed to take great pride in having deliberately rejected legitimate oil filter test protocols (the kind of tests that have been accurately evaluating oil filters for years and driving R&D for better and better filters). Their attitude appeared to be that the legitimate tests couldn't be trusted because they hadn't been devised by Consumers Union's "independent" "experts".

They lean heavily on their claim that they're trustworthy because they don't accept advertising, which is a neat trick to distract readers from the fact that they're in the business of selling Consumer Reports magazine (and website access, books, services, etc). That makes every issue of CR a sort of cover-to-cover advertisement for CR. When I figured that out sometime in the 1980s, I stopped subscribing. The interesting thing: my luck with choosing and buying products and services has been at least as good without CR as it was with CR.

It's really too bad. If you go to your town's main public library and look at dusty old issues of CR from the '30s through '70s or so, you will probably have my same reaction: "What a shame there's nothing like this today". Though you will have to get past some pretty unbelievable uptightness...I recall an article rating clothes irons that went on for a whole paragraph scolding appliance makers for a decorative embossed curlicue in a chrome side panel on an iron. The curlicue had no effect on the performance or safety of the iron, and didn't increase its cost, but CR nevertheless felt compelled to chide the maker for its whimsey. Bet they were fun at parties!

More recently, CR "tested" headlight bulbs and came to a mix of a few valid conclusions you'd have to be blind and dumb not to reach, and a bunch of conclusions that seemed to come from Planet Bizarroland.

As someone who has 35 years of experience in the automotive field.....CRs reviews have generally been quite accurate. Not perfect. No one is. But well within my level of acceptability...

-Virgil- said:

……………...so the CR headlamp test prefers low beams with no cutoff ………………….

Click to expand...

Are low beams with no cutoff or a soft cutoff inherently inferior to low beams with a sharp cutoff?

SubLGT said:

Are low beams with no cutoff or a soft cutoff inherently inferior to low beams with a sharp cutoff?

Click to expand...

No. There are benefits and drawbacks either way.

SubLGT said:

Are low beams with no cutoff or a soft cutoff inherently inferior to low beams with a sharp cutoff?

Click to expand...

Subjectively, for me low beams with too-sharp a cutoff are uncomfortable to drive with.

GreySave said:

As someone who has 35 years of experience in the automotive field.....CRs reviews have generally been quite accurate. Not perfect. No one is. But well within my level of acceptability...

Click to expand...

I suppose they must be within SOMEONE'S level of acceptability, as they've been around a very long time.



They missed my levels though often enough that I don't accept their findings or opinions. If you just mean the overall reported problems tallies for reliability, those ARE probably not too far off at least...because they don't test reliability, they send out a poll and people respond.

TEEJ said:

test reliability, they send out a poll and people respond.

Click to expand...

Shades of "Dewey Defeats Truman". Perhaps the people who subscribe to CR have enough income to properly maintain their cars, or something.

Alaric Darconville said:

Shades of "Dewey Defeats Truman". Perhaps the people who subscribe to CR have enough income to properly maintain their cars, or something.

Click to expand...

Dewey DID defeat Truman, I saw the head line! :devil:


Proper maintenance is one thing, expectations are another. People who's budgets are so tight that the car is totaled if it runs out of gas don't subscribe to Consumer Reports, Nylon Guy, maybe.


So, yeah, a poll can be mis-designed too; have a built in bias, too vague an instruction set, perceived default answers, and so forth.

So, that can skew the data.

I would say though, that overall...CU's reliability ratings are not horrifically useless. Grains of salt should be liberally applied, but the overall tendencies seem to at least not be too out of line with reality.

Their subjective road test data on the other hand is biased in certain respects that are at least fairly consistant.