(links in this post are indicative only - not affiliated with the sites)
You'll at least want a switch, if you haven't got one already, to just switch the entire assembly off when you're not using it. You can find slide-type switches intended for PCB mounting pretty much anywhere, which would work well for your intended use.
http://www2.conrad.nl/goto.php?artikel=708020
Push/hold buttons are usually a bit more expensive and tend to be more bulky as well (due to the mechanisms required for the toggling behavior).
The next question is whether or not you'll need a minimum resistor value for your LED. If you know the details of your LED, you can use an online LED/resistor calculator. If you don't.. then it's usually 'safe' to presume a forward voltage (Vf) of 3.0 for white and blue, 2.6 for green, and 1.8 for red LEDs. Those are gross estimates established years ago that no longer conform to each and every LED, though - so if you do have exact details, that'd be better. Otherwise, just err on the safe side (as you don't have brightness high on your list).
http://led.linear1.org/1led.wiz
Resistors themselves are a few cents each and, again, can be picked up pretty much anywhere they'd have the other components.
Note that a 2032 button cell itself has some internal resistance and will only discharge at low milliAmp levels. That's why you can usually just hook up any LED, even 1.2V red ones, directly to one of these without the current-limiting resistor requirement.
But if you'd decide to upgrade to 2 AA batteries, checking into the above would be prudent.
For the potentiometer, just pick up a 10mm 'trimmer' potentiometer that takes a spindle.
http://www2.conrad.nl/goto.php?artikel=431958
( exactly article is a 15mm 'vertical' version - look for the 10mm version instead. vertical vs horizontal will depend on how you'd want to wire it. )
Note that this article is a 'closed' potentiometer - that is to say: the track is inside the housing, which is at least dirt-proof and mostly splash proof. There's 'open' models as well, which obviously wouldn't be good candidates for outdoor use.
Then source a disc-type spindle. This lays almost flat against the potentiometer, taking up the least space, while giving quite good control.
http://www2.conrad.nl/goto.php?artikel=426040
There -are- potentiometers with that type of spindle already included, but they tend to be more expensive; they're easier to source sometimes, though, and don't require manual assembly.
http://www.westfloridacomponents.com/mm5/graphics/00000001/p148.jpg
( Edit -> note that this is an 'open' type, where dirt and such can fairly easily get to the track )
One rather important question is what value potentiometer you'll need. 1M Ohm is typically going to do the trick to dim down to the lowest level (if the LED doesn't just cut off entirely), but may be too difficult to control as the LED's output curve is not linear, while the potentiometer is (there's logarithmic potentiometers, but rarely in that form factor, and often the wrong way around).
If you happen to have any resistors laying about, you can just try and see what light output you get from putting those in series with your LED. Find the one you like best dimmed all the way, and get a potentiometer closest to that value (a typical series is 1, 2.5, 5.. so if you find that a 330k Ohm resistor gives you what you need, a 500k Ohm potentiometer would be your most likely bet).
That's all you'd need for a -very- basic, but slightly energy-wasting, setup. Just put all of those components in series and off you go.
The potentiometer could be glued to the bottom of your battery holder - the switch can probably be glued to the 'flat' side of the potentiometer, making things very compact indeed.
A slightly more advanced option is to use pulse width modulation, so that the light blinks on/off really quickly to dim. This saves energy (although for a 1 LED setup driven from a 3V button cell, that's debatable) as you're no longer just wasting energy through the potentiometer. Unfortunately, any circuit for that is going to be fairly bulky or relatively expensive.
In addition, PWM does not dim well to very low brightness. The reason for that is that PWM always switches between a given brightness (usually full brightness), and no light at all. Even though LEDs can be switched very quickly, at some point you're just going to lose out to constraints of the circuit driving the LED and the best you'll achieve is a very quickly flickering LED. It might look dimmer, but any movement of the light at all will quickly show this flickering in effect.
(
http://www.reuk.co.uk/LED-Dimmer-Circuit.htm )
So although PWM is a superior method as far as battery life goes, I doubt it's worth the bother in what you describe. What is the application, though?
As far as sourcing these components go, newark, radioshack, etc. will all have these things, but typically in varying models. Especially the potentiometer I pointed to - Piher brand - seems to be less popular in the U.S. where other types are favored; except those don't take spindles. Hence the pre-wheeled versions possibly being easier to source.