There are some very cheap battery testers for AA and AAA batteries ... Mine has an analogue moving coil meter and the dial is marked replace , low , and good ... It has an arm that clips across the battery and you just read off the coloured dial ... It puts a load on the battery for testing purposes ... It works and is cheap off Ebay ... It also checks button cells.
As OLD4570 says, it is just a matter of measuring the Voltage on the Li-Ions ... There is a table showing a range of Voltage readings (open circuit) for rechargeable Li-Ion cells ... It gives the percentage capacity left for a given voltage ... For example 4.2V is 100% (full) , 4.0V is 80% , 3.9V is 60% ... It shows voltage readings down to 3.6V empty ... The table is on CPF somewhere ... I made a written copy for personal use though most of the regulars will know it off by heart.
Good Luck.