Here is an article from 2014 by a (group of) mathematicians who used Mathematica extensively in their research describing the problems they had with Mathematica

Here is an article from 2014 by a (group of) mathematicians who used Mathematica extensively in their research describing the problems they had with Mathematica when trying to calculate the determinant of a Matrix containing large integers (on the order of 10k digits). The bug was reported in Mathematica 7 and was still there in Mathematica 10. https://www.ams.org/notices/201410/rnoti-p1249.pdf https://www.ams.org/notices/201410/rnoti-p1249.pdf Back when I was using Mathematica a lot (2005-2010ish) the Mathematica usenet group kept an archive of all errors in Mathematica people had found, but it seems to have disappeared off the Internet. If you browse the [bugs] tag in mathematica.stackexchange.com you'll find a few as well (although admittedly many of the issues there are user errors rather than Mathematica getting it wrong)