The problem is that simply measuring the voltage will not tell you much. I mean, if you charge the battery and you cannot read the full voltage across it, it is almost certainly bad. But without putting a load on the battery, measuring the full output voltage won't tell you much.
The output voltage will be something less than 20 volts. The input impedance of your typical DMM is 11 Megohms. You will pull something like 1.5 microamps out fo the battery when you try to measure. The battery can be all but graveyard dead and it will still report the full voltage.