One reason it's undesirable is that it's invisible. Let's take a green pointer as an example.
A 5mw pointer may have a 100mw IR diode in it, and 5mw of green comes out the end. Now...what if 20mw of IR also came out the same place, by leaking around the side of the crystal? Being invisible, that *could* pose a risk to your eyes.
Additionally, the total output power of that LASER would be 5mw + 20mw. Now, turn that logic around... A 50mw greenie, without an IR filter, may produce only 5mw of green and 45mw of IR. Its total output is 50mw...not false advertising. But, not 50mw of green...
As to the higher temperatures...not really. It's more a matter of what frequencies are absorbed by the material. It's why it's hard to pop a red balloon with a red pointer...it just bounces off. 50mw is 50mw...
-- Chuck Knight