Every LED current source I know of has at least one resistor in it to measure the current.
Mostly with higher power LEDs you can use a sense resistor that is very small in relation to the output power of the LED. Not so true with 20 mA LEDs, although of course you could make such a driver yourself, but it would cost more than the LEDs.
If you are aiming to avoid the efficiency loss of resistors then LM317, or even the improved NCP1117, is definitely not the way to go. You still end up losing at least 2.4 volts to the regulator, which could be a considerable percentage of your input voltage.
The most efficent solution is to use LED driver specifically designed for this application such as AS1109, TLC 5921 and so forth. TI, Maxim, and Austria have a wide range. Mostly these are drivers for 16 or 32 strings that work like shift registers. They are constant current linear regulators. Some also include PWM. They have integrated op-amp and transistor circuit for each LED output, with max current for the entire unit set by a single external resistor. I think they are pretty much only available in surface mount packages. Of course you will need a micro-controller as well in order to take advantage of these. The chips only cost about a dollar for 16 or 32 channels.
Of course this may be entirely too complicated for what you want to do. This is not the kind of thing you buy from Deal Extreme. You would have to make it yourself.