Anything harder then it can scratch it(not much can though).
It also slowly wears though use. IIRC Aluminum Oxide is slightly soluble so of left in a large enough body of water, water long enough it will dissolve.
The metal underneath it is weaker then the coating so the metal can bend enough so it breaks the HA surface.
It can also chip.
If you look at pictures of well used lights, you will see that it starts to wear around the edges first. That is where the coating is the weakest.
You really have to beat it up a bit to get it scratched: if you EDC a type III anodized light in a pocket without sharp metal objects, its going to stay very clean. Anodizing quality varies between lights, depending on -- among other things -- the purity of the base aluminum.
It's just a strong finish, not a perfect one. HA will get damaged. Most often it will wear off on raised surfaces, edges etc. It doesn't necessarily take something extremely hard, either. A hard anodised light that is carried with keys for a long time will eventually show signs of wear.