Do we have actual data for this other then the usually wrong common sense derived from our cultural values? Most drugs are relatively harmless if taken rarely

Do we have actual data for this other then the usually wrong common sense derived from our cultural values? Most drugs are relatively harmless if taken rarely and in relatively low doses, the problem is that a lot of users don't and that because of our shared cultural history Alcohol typically gets a pass where as things not alcohol is vilified even if the data(as is the case for Cannabis) suggest that they might be more equal in actual harm done.