First, are we talking 9v primary lithiums or rechargeable lithiums? ZTS makes the MBT1 which they say will test alkaline and carbon-zinc 9v batteries. I can't see why it shouldn't give a good approximation for a 9v lithium primary as well. I wouldn't try it with rechargeables though, since the voltages are enough different that one couldn't trust the results.
If you are going to use a regular voltmeter, instead of just using some random resistor, I would make up a test cable that would be placed between the battery and its normal connector, with test points to attach the voltmeter to. That way one would be reading the voltage under the actual load that the battery faces.