I would say it is legitimately much more difficult. Pretty much since the conception of quantum mechanics up until around year 2000 most luminaries of the field

I would say it is legitimately much more difficult. Pretty much since the conception of quantum mechanics up until around year 2000 most luminaries of the field would say "it is impossible to harness single-particle quantum effects, all human tech would be based on collective behavior". And to realize a quantum advantage you need those single-particle effects. Kinda like asking Babbage to build the analytical engine before the wheel (let alone the cog) was invented. Another thing to keep in mind: Creation of a working quantum computer would be a monumental achievement, letting humans harness the full computational power of nature's laws, a defining moment in a civilizations technical capabilities, etc. However, there are only a handful of computational problems at which quantum computers would be exponentially better than classical (most notably simulating novel materials and drugs and certain optimization problems). While these problems are simply impossible on classical computers, for the majority of other problems classical and quantum computers would be on equal footing.