Portside

(CNN) -- Over the last year, I have been working on a new documentary called "Weed." The title "Weed" may sound cavalier, but the content is not.

I traveled around the world to interview medical leaders, experts, growers and patients. I spoke candidly to them, asking tough questions. What I found was stunning.

Long before I began this project, I had steadily reviewed the scientific literature on medical marijuana from the United States and thought it was fairly unimpressive. Reading these papers five years ago, it was hard to make a case for medicinal marijuana. I even wrote about this in a TIME magazine article, back in 2009, titled "Why I would Vote No on Pot."

Well, I am here to apologize.

I apologize because I didn't look hard enough, until now. I didn't look far enough. I didn't review papers from smaller labs in other countries doing some remarkable research, and I was too dismissive of the loud chorus of legitimate patients whose symptoms improved on cannabis.

Instead, I lumped them with the high-visibility malingerers, just looking to get high. I mistakenly believed the Drug Enforcement Agency listed marijuana as a schedule 1 substance because of sound scientific proof. Surely, they must have quality reasoning as to why marijuana is in the category of the most dangerous drugs that have "no accepted medicinal use and a high potential for abuse."

They didn't have the science to support that claim, and I now know that when it comes to marijuana neither of those things are true. It doesn't have a high potential for abuse, and there are very legitimate medical applications. In fact, sometimes marijuana is the only thing that works. Take the case of Charlotte Figi, who I met in Colorado. She started having seizures soon after birth. By age 3, she was having 300 a week, despite being on seven different medications. Medical marijuana has calmed her brain, limiting her seizures to 2 or 3 per month.

I have seen more patients like Charlotte first hand, spent time with them and come to the realization that it is irresponsible not to provide the best care we can as a medical community, care that could involve marijuana.

We have been terribly and systematically misled for nearly 70 years in the United States, and I apologize for my own role in that.

I hope this article and upcoming documentary will help set the record straight.

On August 14, 1970, the Assistant Secretary of Health, Dr. Roger O. Egeberg wrote a letter recommending the plant, marijuana, be classified as a schedule 1 substance, and it has remained that way for nearly 45 years. My research started with a careful reading of that decades old letter. What I found was unsettling. Egeberg had carefully chosen his words:

"Since there is still a considerable void in our knowledge of the plant and effects of the active drug contained in it, our recommendation is that marijuana be retained within schedule 1 at least until the completion of certain studies now underway to resolve the issue."

Not because of sound science, but because of its absence, marijuana was classified as a schedule 1 substance. Again, the year was 1970. Egeberg mentions studies that are underway, but many were never completed. As my investigation continued, however, I realized Egeberg did in fact have important research already available to him, some of it from more than 25 years earlier.

High risk of abuse

In 1944, New York Mayor Fiorello LaGuardia commissioned research to be performed by the New York Academy of Science. Among their conclusions: they found marijuana did not lead to significant addiction in the medical sense of the word. They also did not find any evidence marijuana led to morphine, heroin or cocaine addiction.

We now know that while estimates vary, marijuana leads to dependence in around 9 to 10% of its adult users. By comparison, cocaine, a schedule 2 substance "with less abuse potential than schedule 1 drugs" hooks 20% of those who use it. Around 25% of heroin users become addicted.

The worst is tobacco, where the number is closer to 30% of smokers, many of whom go on to die because of their addiction.

There is clear evidence that in some people marijuana use can lead to withdrawal symptoms, including insomnia, anxiety and nausea. Even considering this, it is hard to make a case that it has a high potential for abuse. The physical symptoms of marijuana addiction are nothing like those of the other drugs I've mentioned. I have seen the withdrawal from alcohol, and it can be life threatening.

I do want to mention a concern that I think about as a father. Young, developing brains are likely more susceptible to harm from marijuana than adult brains. Some recent studies suggest that regular use in teenage years leads to a permanent decrease in IQ. Other research hints at a possible heightened risk of developing psychosis.

Much in the same way I wouldn't let my own children drink alcohol, I wouldn't permit marijuana until they are adults. If they are adamant about trying marijuana, I will urge them to wait until they're in their mid-20s when their brains are fully developed.

Medical benefit

While investigating, I realized something else quite important. Medical marijuana is not new, and the medical community has been writing about it for a long time. There were in fact hundreds of journal articles, mostly documenting the benefits. Most of those papers, however, were written between the years 1840 and 1930. The papers described the use of medical marijuana to treat "neuralgia, convulsive disorders, emaciation," among other things.

A search through the U.S. National Library of Medicine this past year pulled up nearly 20,000 more recent papers. But the majority were research into the harm of marijuana, such as "Bad trip due to anticholinergic effect of cannabis," or "Cannabis induced pancreatitits" and "Marijuana use and risk of lung cancer."

In my quick running of the numbers, I calculated about 6% of the current U.S. marijuana studies investigate the benefits of medical marijuana. The rest are designed to investigate harm. That imbalance paints a highly distorted picture.

The challenges of marijuana research

To do studies on marijuana in the United States today, you need two important things.

First of all, you need marijuana. And marijuana is illegal. You see the problem. Scientists can get research marijuana from a special farm in Mississippi, which is astonishingly located in the middle of the Ole Miss campus, but it is challenging. When I visited this year, there was no marijuana being grown.

The second thing you need is approval, and the scientists I interviewed kept reminding me how tedious that can be. While a cancer study may first be evaluated by the National Cancer Institute, or a pain study may go through the National Institute for Neurological Disorders, there is one more approval required for marijuana: NIDA, the National Institute on Drug Abuse. It is an organization that has a core mission of studying drug abuse, as opposed to benefit.

Stuck in the middle are the legitimate patients who depend on marijuana as a medicine, oftentimes as their only good option.

Keep in mind that up until 1943, marijuana was part of the United States drug pharmacopeia. One of the conditions for which it was prescribed was neuropathic pain. It is a miserable pain that's tough to treat. My own patients have described it as "lancinating, burning and a barrage of pins and needles." While marijuana has long been documented to be effective for this awful pain, the most common medications prescribed today come from the poppy plant, including morphine, oxycodone and dilaudid.

Here is the problem. Most of these medications don't work very well for this kind of pain, and tolerance is a real problem.

Most frightening to me is that someone dies in the United States every 19 minutes from a prescription drug overdose, mostly accidental. Every 19 minutes. It is a horrifying statistic. As much as I searched, I could not find a documented case of death from marijuana overdose.

It is perhaps no surprise then that 76% of physicians recently surveyed said they would approve the use of marijuana to help ease a woman's pain from breast cancer.

When marijuana became a schedule 1 substance, there was a request to fill a "void in our knowledge." In the United States, that has been challenging because of the infrastructure surrounding the study of an illegal substance, with a drug abuse organization at the heart of the approval process. And yet, despite the hurdles, we have made considerable progress that continues today.

Looking forward, I am especially intrigued by studies like those in Spain and Israel looking at the anti-cancer effects of marijuana and its components. I'm intrigued by the neuro-protective study by Lev Meschoulam in Israel, and research in Israel and the United States on whether the drug might help alleviate symptoms of PTSD. I promise to do my part to help, genuinely and honestly, fill the remaining void in our knowledge.

Citizens in 20 states and the District of Columbia have now voted to approve marijuana for medical applications, and more states will be making that choice soon. As for Dr. Roger Egeberg, who wrote that letter in 1970, he passed away 16 years ago.

I wonder what he would think if he were alive today.

Dr. Sanjay Gupta is a neurosurgeon and CNN's chief medical correspondent.