If you're doing C++, you could always try Bloodshed Dev-C++, which is free.
http://www.bloodshed.net/devcpp.html
I haven't used Bloodshed Dev-C++ for a long time, but I had a good impression of it when I used it.
Eclipse SDK
http://eclipse.org is also good, and free. I think it's for developing Java only, but I may be wrong.
If you're not sure what language you want to use, you could consider Perl or Python, which should let you get the job done quickly. Or you could try Scheme (Lisp-like) and try the DrScheme IDE.
I like using gcc for all my compiling needs. It's usually used on Linux and the like, but works on Windows as well
I believe there are quite a few open source programs that will do some calculus already. Eigenmath is an example, but I'm not sure how accurate it is... I tried some things with trig and it came out wrong. Handling matrix equations for my linear algebra class using Eigenmath is a different story. Very handy.
http://eigenmath.sf.net
You might want to look into YACAS (a computer algebra system), although I haven't looked into it very closely yet.
http://yacas.sf.net (I think)