I believe blue light is going to be the most prone to causing retinal damage of the visible wavelengths because the shorter the wavelength, the more potentially damaging it is as the photons that compose the light at shorter wavelengths are more energetic. Also, the eye's sensitivity to blue is quite low, so if the only light source in a room is a blue LED, then the pupils will tend to be dilated, allowing in a greater dosage. However, it is not blue light itself that is the problem as much as high intensity blue light. In nature there really are no blue point sources, as blue light from the sun is attenuated and reaches the earth as evenly dispersed sky light. UV is attenuated even moreso than blue light (the very dangerous high-energy rays are absorbed, and what makes it through will tend to be extremely scattered/diffused as rayleigh scattering of light in the atmosphere is proportional to the inverse fourth power of the wavelength)
The case in which I would see blue LEDs being a problem would looking directly into very poor quality emitters -- the sort that are more "whitish blue" than cool white. Good high-power LEDs that are more of a ~5000K neutral white will not be nearly as much of a problem, as they are so much brighter, the pupil will tend to constrict more if exposed, reducing the received dosage of blue light compared to exposure to a crappy 8000K LED. However, if one were to compare an LED to a HID lamp, or even halogen-incandescent of the same apparent color temperature and intensity, I believe the LED would be safer in almost every case. That is because the LED has almost zero light below about 450nm, whereas HID lights have a strong spike of violet light in the ~420nm, and both HID and halogen-incandescent, if not used with a proper UV-filter, release significant amount of shortwave UV energy. As LEDs move away from nasty blueness toward neutral white, I believe this issue will become largely irrelevant in reality, but that probably won't do anything to stop scare tactics (just look at how many articles there are about CFLs being highly dangerous toxic cesspools of mercury).
This article mentioned UV LEDs intended for use in germicidal lamps as being the most dangerous, and with that I would certainly agree. Germicidal lamps release very short wavelength UV, the variety that causes sunburns. Exposure to that sort of wavelength from a point-source is far more dangerous than even pure blue LEDs as the UV rays would be almost completely invisible (no blink reflex and no pupil response is worse than minimal pupil response) to eyes and much more damaging -- similar to arc-welding without proper protection but without a bright arc to cause a blink reflex.
I certainly hope that damage caused by UV LEDs in weird cases like that is not used to try to scare people away from using perfectly safe white LEDs in flashlights and household lighting. There is no reason LEDs cannot be made at least as safe or safer than conventional incandscent and fluorsecent lights.
Ultimately though, the danger comes from a combination of blue/violet/UV light and high intensity, while I am very wary of using UV LEDs higher-powered than 10mA coin cells used to check $20 bills, I would not worry at all about using a fluorescent black-light at the same wavelength as the radiation is diffused in that case.
