I would just start w/ python, but I know several people who are in love w/ R http://www.r-project.org/ http://www.r-project.org/ which is deigned for statistics

I would just start w/ python, but I know several people who are in love w/ R http://www.r-project.org/ http://www.r-project.org/ which is deigned for statistics (there is an RPython interface, so you can tally your cake and have it too :)