There's a real problem of getting reliable information on whether drugs are as dangerous as the law suggests. Who do you trust?
The politicians who make drug laws don't use drugs, at least not publically. Their information comes second and third hand at best, and relies largely on research by third parties. Their goals include raising their own personal profile and keeping important people happy at the cost of others. They're at least occasionally corrupt.
The scientists only know about drugs what they can verify using science, and even then only when paid. Even then, the results are only as public as public interest. The meaning of the results can and will be twisted by people who aren't scientists and have their own biases. Studies will frequently be funded by anti-drug campaigners to lend credence to their message, but not government groups (who assume they already know all the details) and certainly not drug users (who may not have funding and, as persons who break the law, cannot act openly). Again, because drug use is illegal, research is hampered, and can only take place with government consent, which only occurs when government has its own agenda or else accedes to someone's demands. Another setback is that science knows relatively little about the functioning of the brain, making any evidence that would exonerate drug users extremely costly and time-consuming to extract.
The police only enforce the law, even if they don't agree with it. They may have a skewed perspective as they are responsible for managing drug-related crime.
The average person knows little about drugs, and isn't qualified as an authority on the topic. If they do use drugs, they do so in secret, as it's illegal. This discourages information flow on the topic and encourages the public consensus to support prohibition.
The drug users themselves are close enough to drugs to understand them better than anyone. However, their authority is undermined at several points. They may be expected to lie to protect themselves. Opponents can claim that the effect of drugs on the mind renders the opinion of users meaningless - they become a person without authority, like a child or madman. The only set of universally trusted drug users are strongly reformed ex-users, and this group has a clear bias.