"For example, prosecution of individuals with cancer or other serious illnesses who use marijuana as part of a recommended treatment regimen consistent with applicable state law, or those caregivers in clear and unambiguous compliance with existing state law who provide such individuals with marijuana, is unlikely to be an efficient use of limited federal resources," the memo states.
Proponents of medical marijuana say the new policy is long overdue.
"What you have now, finally, after all these years is a federal government, which is not at war with its own states and not at war with suffering sick people in these states," Bruce Mirken, communications director for the Marijuana Policy Project, told ABC News. "It is the most positive and significant development on federal medical marijuana policy since the Carter administration."
But opponents have expressed concern that such a policy could also lead to more illegal drug use and sales. According to the Justice Department, marijuana distribution in the United States remains the single largest source of revenue for Mexican cartels.
Rep. Lamar Smith, R-Texas, the ranking member on the House Judiciary Committee, expressed disapproval of the new guidelines, saying that it weakens federal enforcement of drug laws.
"The administration's new guidelines directing federal prosecutors to ignore local medical marijuana dispensaries that allegedly operate in compliance with state laws fly in the face of Supreme Court precedent and undermine federal laws that prohibit the distribution and use of marijuana," Lamar said in a statement. "By directing federal law enforcement officers to ignore federal drug laws, the administration is tacitly condoning the use of marijuana in the U.S. If we want to win the war on drugs, federal prosecutors have a responsibility to investigate and prosecute all medical marijuana dispensaries and not just those that are merely fronts for illegal marijuana distribution."
In March, when Holder first broached the subject, Republican Iowa Sen. Chuck Grassley blasted the administration, saying that marijuana use leads to use of harder drugs.
"The first rule of medicine, first do no harm, is being violated by the attorney general by his decision," Grassley said.
Others fear that marijuana dispensaries will also be used to sell drugs to not just medical patients, but to recreational users as well. There's also the question of what will happen to those cases that are already pending in courts.
Citing that the president has said he'd like to change how the cases are handled, one official said the guidelines do not constitute a new policy but rather it gives prosecutors guidance on how to handle cases.
But it does reflect a clear shift away from the previous administration's policies, and some proponents hope it will pave the way for more states to relax their marijuana laws.
"The last couple of years, it's been so hard because I've felt so guilty every time someone got busted. ... I felt guilty because I lost my case in the Supreme Court," said Raich, who will undergo brain surgery later this month. "We have work to do still. What's going to happen to patients in those states where it's not legal? It's bittersweet."
ABC News' Jason Ryan contributed to this report.