"Rex and I have been talking about this for a long time. We got along actually quite well but we disagreed on things," Trump said Tuesday morning.
"When you look at the Iran Deal. I thought it was terrible. He thought it was okay," Trump said. "I wanted to either break it or do something, he felt a little differently. So we were not really thinking the same."
But Tillerson was not alone in his view that the international agreement - which restricts Iran's enrichment of uranium and its stockpiles - should stay.
"If we can confirm that Iran is living by the agreement, if we can determine that this is in our best interest, then clearly, we should stay with it," Mattis said. "I believe, at this point in time, absent indications to the contrary, it is something the president should consider staying with."
Trump, in contrast, has repeatedly called it one of the "worst deals" in U.S. history, and in January threatened to withdraw the U.S. unless European allies worked with the administration to sanction Iran's ballistic missile program and combat its other "malign" behavior.
Mattis' view backs up those of his top generals, including the head of U.S. Central Command, who told senators on Tuesday that the deal is in America's national security interests, saying it "addresses one of the principal threats that we deal with from Iran."
In September, the chairman of the Joint Chiefs of Staff, Gen. Joseph Dunford, told Congress that the intelligence community's assessment was that Iran was in compliance with the deal, and therefore the U.S. should "focus on addressing the other challenges" posed by Iran.
Dunford warned that "holding up agreements that we have signed, unless there's a material breach, would have an impact on others' willingness to sign agreements."
Trump and Tillerson disagreed on several top foreign policy issues beyond the Iran deal, including North Korea, the Paris Climate accord, and multilateral trade deals like NAFTA and TPP.
But the differences became personal when, in October, Tillerson would not directly deny reports that he called the president a “moron.” Trump shot back with allegations that he was smarter than his top diplomat, saying the two should "compare IQ tests."
Fourteen months into the Trump administration, Mattis has avoided the scrutiny from his boss directed at so many of his colleagues.
If Tillerson wasn't in lockstep with Trump, Mattis has given off the impression that the two had a close relationship and were often allies. Most recently, both Tillerson and Mattis opposed tariffs on steel and aluminum.
In January, Mattis said he spoke with Tillerson "two, three times a day" and shared weekly breakfasts.
Now, Mattis will adopt that working relationship with Pompeo, with whom he already interacts frequently as CIA director.