That's about it in essence. I have a gen1 Touch, and it doesn't have a speaker like the current models. Any app that doesn't use iPhone specific actions, like dialing, talking, etc. generally works on the Touch. The App store should say if there there is a compatibility problem - otherwise, unless it's something that obviously does some function the touch doesn't have, assume it will work.
You can't access the net over the cellular, so you miss out on slow but always-available-when-the-phone-works service. Makes you a wifi whore. Note that the newest 32/64GB models are faster and more featured than the 8GB. Don't pay full for an old 32.
Like you say, it good if you have a phone you like already or don't want ATT service and can live with wifi only.