Maybe.
Is it this driver?
http://www.shiningbeam.com/servlet/the-132/3-dsh-Mode-Regulated-Circuit-Board/Detail
So, on high that driver should pull 2.8 amps from your batteries. I don't know what you are using, hopefully they are quality primary cells, not made in china. Let's take for instance a set of Duracell CR123A. Here is the data sheet:
http://lib.store.yahoo.net/lib/theshorelinemarket/DURACELL-CR123A-ULTRA-DATASHEET.pdf
They are rated as 1550 mAh but, as you can see from the continous discharge graph, that is at 1000mA draw. Now, look at this
graph so kindly provided by SilverFox. It is located a bit more than halfway down the page. You can see at 2.5A he is only getting ~22 minutes of runtime, and less than 1Ah.
If you only look at the label you see 1550 mAH and you figure OK, I am pulling less than 3 amps, ergo, less than a 2C load. 2C is fine, though it is getting borderline. But, at 22 minutes we see in reality this is a 3C load. And that is at 2.5A but your draw is actually a bit higher.
The most obvious giveaway is that you say they are getting hot. Lithium cells shouldn't heat up during normal usage, so even if you were within safe boundries I would still be concerned. Perhaps though the led heating the flashlight body could be heating the batteries, but, I doubt it.
In addition, the resting voltage of 2 primary CR123s is probably closer to 6.6V, a bit high for that driver. But, at high current I am sure the voltage sags more than enough to not be concerned about it.
If you are going to use a 7135 based linear regulator (that is what you have) then your best battery solution is a single cell, 18650 or larger. You won't get a ton of runtime before falling out of regulation, but it is safe.