Yes, exactly. I'm probably not being very clear by trying to be brief. Approximately half of the light from the die (before or after down conversion) will be back scattered and re-absorbed by the die surface. Whatever else makes it back will suffer the same odds, plus the light is likely to be down converted to a wavelength not absorbed by the phosphors so it will be scattered (minus Fresnel loss). There are a ton of papers describing the efficiencies of phosphor down conversion and die modeling. This is a quick one that talks about back scatter and loss: http://lightingresearch.org/programs/solidstate/pdf/Zhu-SPIE6337.pdf