A hugely comprehensive program, which seems to integrate well with all sorts of open- (and closed-) source CASs, numerical routines, etc. is Sage: http://www.sagemath.org/
I've played with it a bit, and have always ended up going back to Mathematica. However, it really is a nice project, if perhaps a bit bloated at the moment.
I used Scilab (mentioned below and previously) to do some pretty heavy lifting for a data-analysis project. It worked admirably, and very quickly. I hadn't used Matlab much before it, but was able to figure out the syntax well enough to get the job done.
I am particularly recently enamoured with Python as a scientific language. I have never been one to jump on a bandwagon, especially when the riders are so obviously caught up in a religious ecstacy, and have stuck with some unpopular software alternatives simply out of stubbornness. However, having sampled Python, and having easily figured out how to do some things with it that took me -- literally -- years with Mathematica, I am coming around to being a convert of sorts. If one is looking for a good starting language, or a pedagogically "easy" one for, e.g., teaching students some fundamentals, it's worth a serious look.