That might've been your personal experience. But the Defense Language Institute, which has trained thousands of people over the decades, gives both Chinese and

That might've been your personal experience. But the Defense Language Institute, which has trained thousands of people over the decades, gives both Chinese and Japanese (along with Arabic and Korean) a level "4" rating - the highest one, requiring (IIRC) 2200 hours of practice. A large part of it depends on the individual - if you find grammar easy to pick up, then Japanese will be easier. The opposite will be true if you find pronunciation easier.