I have used 3mm LEDs in various colors over the last 15 years, all direct drive (2AA, usually in minimags), even the 2.5v green ones. I find that although light output doesn't go up much with higher voltages on red and green, they usually take anything less than 50ma current with no problems. It should shorten the life of the LED and reduce run time, but I haven't noticed. After a few days use, I simply put fresh batteries in, whether it needs it or not. Otherwise it just keeps getting dimmer and dimmer. I found one I left in my pack for about a month. It was pretty dim. This is a good way to drain the last erg from batteries.
The white LEDs usually don't give full output on 2 batteries, even lithiums, but they put out usable amounts of light. With 3mm, the leads fit the regular bulb socket (no perf board required). Every once in a while, the reflector will have too small of a hole, which is easily drilled out.
I haven't tried, but if you are going to drill the reflector out, perhaps a 5mm LED would fit....then you might want to remove the 'do not remove' part & use perf board instead. Some LEDs have a fairly wide angle output & get some benefit from the reflector. narrow output LEDs can do without it. Let us know how you do with it.