I'm DD'ing a Q5 off a 18650 li-ion. Current drops in all the real world connections/terminals etc meant that I saw ca 4v only at the LED terminals.
Given the lower voltage of the a123s I doubt you'd have any problems... wont likely get you to max brightness though since many Q5 samples (at least the ones from DX) tend to have a high-ish vF. hth