Mathomatic is a free, portable, general-purpose Computer Algebra System (CAS) that can automatically solve, differentiate, simplify, combine, and compare algebraic equations, perform standard, complex number, modular, and polynomial arithmetic, etc. It does some calculus and is very easy to learn and use. Plotting expressions with gnuplot is also supported (requires gnuplot).
New users: try typing "help examples". There are many interesting Mathomatic scripts in the "tests" directory, that show what Mathomatic can do, and they teach some mathematics, too.