For $28.99, you can get one of the Mini ZTS battery testers from Lighthound that will test the batteries under load. For any application that uses multiple lithium ion batteries, it makes a very good way to be sure that cells are matched and less likely to cause an explosion.
The mini ZTS doesn't test the rechargeable lithion ion batteries that are in the 4.2 to 3.7V range. Its a great little tester but it won't read those batteries. You need the much more expensive model to test those, which the original poster may not want to pay for.
A $5 analog meter will be useless for testing batteries for voltage as a way to indicate level of charge. Far too little resolution.
A digital voltmeter with at least 3 full digits is necessary. Nothing wrong with investing in a good digital multimeter to have around anyway. You can go to some hardware stores and get a halfway decent one in the $20 price range. You can go to radio shack or some other electronics store and buy very cheap a 10 or 20 ohm resistor to use as a load on the battery. You would need to get at least a 5 watt rated resistor for a quick load test per battery.
By using a digital voltmeter with the resistor across the terminals when you test the battery it will put a load on it and give you a more realistic voltage reading of the battery. This is very useful in comparing to other batteries of the same type and brand. If you are checking RCR123's that range from 4.2V under no load to 2.7V when nearly fully discharged (as seen under some level of load), then you can get a better feel for them when you test them with the resistive load. You only need to test them long enough to get a reading, like 3 seconds. If you are using flashlights that take two or three of them in series it would be best to use the ones with the most closely matching voltages or see which ones are not holding a full charge and not use them at all.
Putting a group of batteries on the charger and when they are all done some read 4.15 V and some read 4.05V under the load, you wouldn't want to mix those together. You would want to use in this example all the batteries that were within just a couple of millivolts of 4.15V together and not throw in one that only read 4.05V. If you just charged your batteries and 10 minutes after it cooled down it read 3.87V on the meter under load, its indicating its not holding a full charge, its getting old and weak. May not want to use it at all, or just use it in a single cell light that doesn't put another battery in series with it.
These are just example numbers to give you an idea of how it would work. Not a set of absolute numbers. I do personally have lith ion batteries that read in these ranges that I keep track of and separated.
4.2V under a 10 ohm load is 420 mA of current draw through the resistor, none through the meter. That is not a bad load for a quick 3 second test on rechargeable battery to separate the weak from the strong.
4.2V under a 20 ohm load is 210mA which is also not a bad load. I paid like a buck each for these resistors in a 10 watt rating. A cheap investment.
Also this should be in the battery thread section not in the LED light section.