Double the Watt rating of what you need and you're pretty safe. The problem is that the specified Watts are measured without a load and when you actually start charging something the actual produced Watts are 25% lower plus you'll have a 25% loss for the battery charging process itself. Going bigger than twice the needed power rating assures better performance in less than optimal conditions.
As an example my 4W USB panel measures 0.8A @ 5V with the multimeter on the open connectors which is the advertised 4 Watts but when it charges something it drops to 0.58A @ 5V which is only 2.9W.
Knowing the charging specs of your charger (or devices with a built in charger) is key to get the right panel as this is the limiting factor of how much power can actually be used. The iPhone 4 can only handle a 0.72A @ 5V which is a mere 3.6W, a 27W panel will not fry it but is a bit overdone, 7W should be the minimum and a 10 to 12W panel is perfect to be on the safe side.
It's also advisable to use a regulating batterypack between the panel and the charger to ensure a constant charge on the charger in case a cloud obscures the sun, etc. Be sure to use a pack that supports pass-thru or load balancing, with this feature the power that gets in goes out the other side without charging the internal battery which would result in a 25% loss. When more power comes in than is needed it will store the excess power and when the panel can't deliver enough power it will supplement it for a constant output.
Personally I prefer all my chargers and sources to be compatible with the USB format as it's the smallest most universally used power socket and you can use other sources besides solar like the BioLite stove and PowerPot cooking pot.
This is my off the grid charging solution sance the sources:
The Voltaic V11 pack transforms the 8.4V input from the Nikon EN-EL15 DSLR battery to a 0.9A @ 5V to charge the 18650 3400mAh in a ML-102 USB charger (max 5W) which charges another EN-EL15 battery in a Pixo C-USB (max 4.5W) multi charger (3.7 & 8.4 Li-Ion, AA, AAA). The 18650's are used in my Surefire lights or Spark headlight, they also serve as my backup power (I'll bring 2 to 10 spares depending on the length of the trip).
My big solarpanel is a Voltaic 16.8W panel btw, when using a dual USB car adapter in the 12V socket I can charge two USB chargers at once with ample reserves for less than optimal light conditions.
That said I prefer to bring enough batteries with me to last me the whole trip as the sun doesn't shine that much during hiking season in Northern Europe, with the setup posted above I can use whatever charge I have in my 18650's or DSLR batteries to charge anything else if needed.