During his tenure, Mattis has stressed to Trump the value in America's security alliances, particularly the NATO alliance.
"My views on treating allies with respect and also being clear-eyed about both malign actors and strategic competitors are strongly held and informed by over four decades of immersion in these issues," Mattis wrote. "We must do everything possible to advance an international order that is most conducive to our security, prosperity and values, and we are strengthened in this effort by the solidarity of our alliances."
Mattis hand-delivered the letter to the White House on Thursday during a meeting with President Trump. Upon his return to the Pentagon, he briefed his staff on his decision, chief Pentagon spokesperson Dana White told reporters.
This was not a forced resignation, but a decision Mattis made of his own accord, a U.S. official told ABC News.
"I believe it is right for me to step down from my position," Mattis wrote the president, adding that he would stay on the job through Feb. 28, 2019 to give enough time to transition to a confirmed successor.
He specifically expressed concern about China and Russia, nations who he said have made clear they "want to shape a world consistent with their authoritarian model."
Mattis added that he believes the U.S. "...must be resolute and unambiguous in our approach to those countries whose strategic interests are increasingly in tension with ours.
His announcement comes a little over two years after Trump nominated him to the position. It also comes a day after Trump's surprise announcement that he was pulling U.S. troops out of Syria and the same day that news broke that the administration was weighing a significant reduction in the 14,000 U.S. troops in Afghanistan.
Mattis retired from the Marine Corps in 2013 after a 41-year career in which he led troops in Kuwait during the Persian Gulf War, in Afghanistan during the initial U.S. wave in 2001 and in Iraq during the 2003 invasion. He capped his career as the head of U.S. Central Command, where he was in charge of all American forces serving in the Middle East and oversaw the wars in Iraq and Afghanistan.
Top Republicans and Democrats reacted to news of the defense secretary's resignation with alarm Thursday night, expressing concern about the chaos in the administration and the message his departure sends to the world.
"What happened in Syria is the straw that broke the camel’s back," Rep. Eliot Engel, D-New York, the incoming chair of the House Foreign Affairs Committee told reporters. "It’s a terrible move, it will clearly backfire."
"I quite frankly slept better knowing he was in charge of our military," Rep. Michael McCaul, R-Texas, the chairman of the House Homeland Security Committee, said. "We want our enemies to fear us, and we want our allies to trust us. They trusted Secretary Mattis and he stabilized world view and world opinions so I think that’s going to be the loss."
Rep. Ruben Gallego, D-AZ, a Marine veteran and member of the House Armed Services Committee, said it speaks volumes about the dysfunction in the White House that a decorated Marine like Mattis determined "that he can no longer be a functional part of the government,"
"The best thing he could do is to be able to expose how dysfunctional the administration is," he said.
In a tweet, Sen. Lindsey Graham, R-South Carolina, said "It is with great sadness that I was informed of the resignation of General Mattis. He is one of the great military leaders in American history. He should be proud of the service he has rendered to President @realDonaldTrump and our nation."
Senate Majority Leader Mitch McConnell released a statement, urging Trump to select a leader who shared Mattis' understanding of our friends and foes.
"I believe it’s essential that the United States maintain and strengthen the post-World War II alliances that have been carefully built by leaders in both parties. We must also maintain a clear-eyed understanding of our friends and foes, and recognize that nations like Russia are among the latter. So I was sorry to learn that Secretary Mattis, who shares those clear principles, will soon depart the administration. But I am particularly distressed that he is resigning due to sharp differences with the president on these and other key aspects of America’s global leadership, according to McConnell's statement.
"It is regrettable that the president must now choose a new Secretary of Defense. But I urge him to select a leader who shares Secretary Mattis’s understanding of these vital principles and his total commitment to America’s servicemembers," the statement continued.
Reports have circulated for months that Mattis could be the next senior Trump administration official to leave office, especially as he has been seen as curbing the president's more impulsive military and personnel choices.
Early on, President Trump constantly referred to his Defense Secretary as "Mad Dog" a nickname from his military years that Mattis actually did not like.
Inexperienced in foreign affairs Trump relied heavily during the early part of his administration on Mattis' experience. Last summer, the Defense Secretary convinced Trump to keep U.S. troops in Afghanistan after the president had suggested pulling completely out of Afghanistan.
But over the past year there have been a number of instances where the President has announced policies that were at odds with Mattis' advice.
They include the suspension of U.S. military exercises with South Korea following Trump's summit meeting with North Korean leader Kim Jong Un, the deployment of active duty troops to the southwest border and the withdrawal of U.S. troops from Syria.
The President's lack of enthusiasm for his Defense Secretary became apparent during a 60 Minutes interview in October.
“I think he’s sort of a Democrat if you wanna know the truth,” Trump said. “But General Mattis is a good guy. We get along very well. He may leave. I mean, at some point, everybody leaves. Everybody. People leave. That’s Washington.”
Asked at the Reagan National Defense Forum earlier in December if he had any plans to leave office, the secretary instead spoke about his love for U.S. troops and revealed once again a deep-seated duty to serve the country.
"When the President of the United States asks you to do something in America, you just do it. To quote Nike, 'Just Do It,'" Mattis told an audience at the Reagan Presidential Library in Simi Valley, California.
With his departure, Mattis will become the last of the three military generals serving in the Trump administration that the president once referred to as "my generals".
Army Lt. Gen. Henry McMaster stepped down as National Security Adviser in March and former Marine General John Kelly, the White House Chief of Staff, has already announced that he will be leaving his post at the end of the year.
ABC News' Ben Siegel contributed to this report.