"While I will forever be grateful for the trust and confidence President Trump has placed in me by giving me this opportunity, I am regretfully withdrawing my nomination to be Secretary for the Department of Veterans Affairs," Jackson said in a statement issued by the White House this morning.
Jackson's nomination has been dogged in recent days by allegations that he improperly dispensed medications and that he wrecked a government vehicle after drinking at a Secret Service farewell party.
Jackson is the presidential physician, having served in the role since President George W. Bush's administration.
"It has been my distinct honor and privilege to work at the White House and serve three Presidents," Jackson said. "Going into this process, I expected tough questions about how to best care for our veterans, but I did not expect to have to dignify baseless and anonymous attacks on my character and integrity."
Jackson, 50, goes on to rebut the allegations that have been leveled against him as "false and fabricated."
"The allegations against me are completely false and fabricated. If they had any merit, I would not have been selected, promoted and entrusted to serve in such a sensitive and important role as physician to three presidents over the past 12 years," he said.
In a morning interview with Fox News minutes after Jackson's withdrawal became official, President Donald Trump defended Jackson against what he said were all false allegations.
“He would have a done a great job,” Trump said. “Tremendous heart. These are all false accusations. These are false. They are trying to destroy a man.”
The president went on to suggest there will be a political price to pay for Democratic Sen. Jon Tester of Montana, whose office has been vetting Jackson and raising questions about the allegations against him.
“I want to tell you, Jon Tester, I think this is going to cause him a lot of problems in his state," Trump said. "For Jon tester to bring up stuff like candy man and kinds of things he was saying.
“Well, you know, these are statements that are made. There is no proof of this. I think Jon Tester has to have a big price to pay in Montana.”
With Jackson’s stepping aside, the president said he has someone in mind to fill the gap but wouldn’t reveal who except to say the person has political experience.
And though Jackson has withdrawn as the nominee to lead the V.A., White House press secretary Sarah Sanders said he is still on the job as the White House doctor.
"Admiral Jackson is a doctor in the United States Navy assigned to the White House and is here at work today," Sanders said in the statement.