Most likely normal, cant say about that specific charger, but on PWM circuits in general, under light load like that, it may go into "pulse skipping" where rather then a constant 50+ khz frequency, the SMPS stops oscilating as long as the voltage is within tolerance. This off-on cycling may occur in the audible band, making the noise you hear.
Some flashlights do something similar to this, if the driver runs at some inaudible frequency, but the PWM dimming runs at 500 hz (plenty fast for there to not be flicker) you could hear the inductor or other components humming, usually a bit louder as you dim the light.
My cell phone charger also hisses some when its hooked up to the phone and charging is complete too, now that I think of it.