Yeah, the lens will come off if you push it too hard.
I know it will withstand 50G crash shock and not phase it (tested), and 1600 G's according to CREE's testing.
I've mounted the LED in a metal box and had people throw it at a wall, many times. The lens stays intact.
When you push or pull on it, you are *seriously* abusing the poor thing.
It is in fact glass.
I noticed what appears to be bubbles underneath the silicone gel in your LED. If this is in fact the case, the company that mounted the CREE to the MCPCB either didn't de-humidify the LED properly before reflow, or they overheated it. Not good, not good at all.
The adhesion of the lens can be quite compromised by improper use of pick and place machines, which would be used by the company that mounted the CREE LED to the MCPCB.