In his exclusive interview with ABC News' chief anchor George Stephanopoulos that aired Sunday night ahead of the April 17 release of his book "A Higher Loyalty: Truth, Lies, and Leadership," Comey talked about the possibility of the president obstructing justice, his concerns over whether the Russians have something to hold over Trump, the way he defends his actions relating to the Hillary Clinton email saga, and much more.
Here is a rundown of the top moments from the interview.
He was nervous when he was about to meet Trump
It wasn't until January 2017, during the presidential transition, that Comey met Trump in person.
"I'm about to meet with a person who doesn't know me, who's just been elected president of the United States ... from my watching him during the … campaign, could be volatile," Comey said. "And I'm about to talk to him about allegations that he was involved with prostitutes in Moscow and that the Russians taped it and have leverage over him."
Comey is breaking his long silence ahead of the April 17 release of his book "A Higher Loyalty: Truth, Lies, and Leadership."
He said that at least on a physical level, Trump lived up to Comey's expectations.
"He had impressively coiffed hair … it looks to be all his. I confess I stared at it pretty closely and my reaction was ‘it must take a lot of time in the morning," Comey said.
"His tie was too long as it always is ... he looked slightly orange up close with small white half-moons under his eyes, which I assume are from tanning goggles," he said.
One of the main purposes of the meeting was to talk to the incoming president and his team about Russian meddling in the election.
The move struck Comey not only because it was atypical to involve intelligence agency heads in those discussions, but because it also made him think of his past work prosecuting mafia bosses. He said it felt as if Trump and his team were trying to make it feel like Comey and the other intelligence heads there were all working as part of the same team – or in mafia terms – part of the family.
"We're all part of the messaging, we're all part of the effort. The boss is at the head of the table and we're gonna figure out together how to do this," Comey said.
Comey said he believed it's "very strange" to be comparing the then-incoming president to a mob boss, but it was a comparison he included in his book.
"I'm not trying to … suggest that President Trump is out breaking legs and shaking down shopkeepers. But instead, what I'm talking about is that leadership culture constantly comes back to me when I think about my experience with the Trump administration,” Comey said.
Trump and his team have repeatedly attacked Comey's credibility, both previously and in the lead up to tonight's interview.
Just today, Trump called Comey "slippery" and "not smart," adding that he "will go down as the WORST FBI Director in history, by far!"
Detailing the "dossier"
Russian involvement in the election was not the only topic of conversation when Comey spoke to Trump in that transition meeting in January.
He also discussed the “dossier,” a 35-page document containing raw intelligence compiled by former British intelligence agent Christopher Steele.
Comey said he didn't talk to him about the financing of the "dossier," and even told Stephanopoulos that he doesn't recall using the term "Steele dossier" at all. One of the more salacious topics covered in the "dossier" relates to unverified claims -- which have been denied by Trump since -- that he had been involved with prostitutes during a 2013 trip to Moscow.
Comey said he was "as graphic as I needed to be" in describing the claims, noting how he didn't get "into the business about people peeing on each other.”
"He interrupted, started talking about it, you know, 'Do I look like a guy who needs hookers?' I didn't answer that, and I just moved on and explained: 'Sir, I'm not saying that we credit this, I'm not saying we believe it. We just thought it very important that you know,”" Comey said.
When asked by Stephanopoulos if he believed Trump's denial, Comey said he isn't sure to this day.
"I honestly never thought these words would come out of my mouth, but I don't know whether the current president of the United States was with prostitutes peeing on each other in Moscow in 2013,” Comey said. “It's possible, but I don't know.”
Email saga presented "no-win situation"
Comey said he knew the investigation into former Secretary of State Hillary Clinton's use of a private email server was going to present a "no-win situation" for him.
"One half of the partisan divide is gonna be angry at us no matter what we do," he told Stephanopoulos.
The email saga became a major part of the 2016 presidential campaign, as Clinton was the Democratic candidate.
Clinton opponents, including Trump, regularly criticized Comey and investigators for choosing not to bring Clinton before a grand jury, but Comey defended that decision in the ABC News special.
"We would prefer with a subject of an investigation to do an informal interview. Lot more flexibility there," Comey said, adding that "they're still required to tell the truth."
He also defended his actions because he viewed it as an effort to defend the independence of the FBI and its investigation. He added that there were certain concerns about then-Attorney General Loretta Lynch, who reportedly asked him to call the email saga a "matter" and not an "investigation."
"It worried me. It gave me an uncomfortable feeling because the Clinton campaign had been trying to come up with other words to describe it," he said.
"I decided I have to step -- as much as I like her, I have to step away from her and show the American people the FBI's work separately," Comey said of Lynch.
In the end, Comey believes the investigation achieved its goals.
"The FBI drove this investigation and we did it in a competent and independent way. I would bet my life on that," he said.
Many took issue with the way the investigation was handled, especially in the wake of later revelations that the FBI had opened an investigation "into whether there were any Americans associated in any way with the Trump campaign who were working with Russia as part of Russia's effort to influence our election," Comey said.
Stephanopoulos pointed to the argument that many said Comey was operating under a double standard, detailing so much about the investigation into Clinton's email investigation and not revealing the investigation into Trump associates.
"Take a step back and stare at the two cases and the posture they were in. The Hillary Clinton email case, was public, and the counterintelligence investigations trying to figure out whether a small group of people, not Donald Trump -- we were not investigating Donald Trump -- whether this small group of Americans was coordinating anything with the Russians,” Comey said.
“We had just started the investigation,” he added. “Didn't know whether we had anything. So it would have been brutally unfair to those people to talk about it. And it woulda jeopardized the investigation."
When asked if he would still send the Oct. 28 letter -- which announced that the FBI was looking back into the Clinton email investigation just days before the election because of the emails found on former Congressman Anthony Weiner's computer -- if he knew that the letter would elect Trump, Comey said he would.
"Down that path lies the death of the FBI as an independent force in American life. If I ever start considering whose political fortunes will be affected by a decision, we're done. We're just another player ... in the tribal battle," Comey said.
As for what he would say to Clinton, who has written that she felt Comey "shivved" her with his actions, Comey said he would stick to his decisions.
"I would hope both camps will read this and, I hope, see a deeply flawed human surrounded by other flawed humans trying to make decisions with an eye, not on politics, but on those higher values," he said.
Surrounded by Clinton supporters
Comey revealed that while he didn't vote in the 2016 election -- saying that as the FBI director he was "trying to be outside of politics" – there were a lot of Clinton supporters in his family.
"I'm pretty sure that at least my four daughters, probably all five of my kids, wanted Hillary Clinton to be the first woman president. I know my amazing spouse did," he said.
Patrice Comey, his wife of more than three decades, told Stephanopoulos that she was a strong supporter of Clinton's.
"I wanted a woman president really badly, and I supported Hillary Clinton. A lot of my friends worked for her. And I was devastated when she lost," she said.
Patrice Comey and the couple's daughters participated in the Women's March, which was held the day after the inauguration.
An awkward public interaction with Trump
Comey was invited to the White House for an event with law enforcement officers, but he was hesitant.
"I worried very much about that independence of the FBI. And, remember there were a whole lot of people who thought that I had helped elect Donald Trump and so why would I wanna go to a public reception with the president two days after he takes office?" Comey said.
He decided to go, and thought he was potentially going to be overlooked because his suit was roughly the same color of the blue curtain, but that didn't happen.
Trump called him over.
"I remember the walk ... it seems like 1,000 yards across the blue room, the oval Blue Room … and my family’s had a lot of fun watching my face as I walk across … ‘cause they know that’s my ‘Oh no’ face," Comey said, going on to note how, among family, that look is known as his "oh s---" face.
Dinner with a deal on the side
Days after that Oval Office photo opportunity, Trump called Comey and invited him to dinner at the White House.
"He said, 'You know, a lot of people would wanna be FBI director and, given all you've gone through, I would understand if you wanna walk away. But it would look like you'd done something wrong if you did that. But I figured I should meet with you and … see what you wanna do,'" Comey said.
"He said, 'I expect loyalty, I need loyalty.' And I just stared at him and had this little narrative with myself inside, saying, 'Don't you move, don't you dare move. Don't even blink,'" Comey said.
"I think because I was caught totally by surprise. I couldn't think of a clever response. And by the second time he came back to it, I had my wits about me and had a better answer," Comey said.
"He said, again, 'I need loyalty.' And I said, 'You will always get honesty from me.' And he paused and then he said, 'Honest loyalty,' as if he was proposing some compromise or a deal. And I paused and said, 'You'll get that from me,'" Comey recounted.
Reading into Trump's alleged request
Another uncomfortable moment came on Feb. 14, 2017, the day after then-National Security Adviser Michael Flynn had been forced out. After a security briefing, Comey said that Trump asked everyone else to leave, including Vice President Mike Pence and Attorney General Jeff Sessions, who was Comey's boss at the time.
"It's so unusual for the FBI director and the president to be alone at all. But to kick out the vice president of the United States and the attorney general, who I work for, so you could talk to me alone, something was up," Comey said.
Comey said Trump "said he hopes I can let it go," referring to the criminal investigation into Flynn.
"His words were, though, 'I hope you can let it go.' I took the expression of hope as, 'This is what I want you to do,'" he told Stephanopoulos.
Trump has denied saying this.
Comey said he didn't stop Trump from making such a request, and told Stephanopoulos "maybe I should have."
"As I've thought about it since, if he didn't know he was doing something improper, why did he kick out the attorney general and the vice president of the United States and the leaders of the intelligence community? I mean, why am I alone if [he] doesn't know the nature of the request? But it's possible that in the moment I shoulda -- you know, another person would have said, 'Sir, you can't ask me that. That's a criminal investigation. That could be obstruction of justice,'" he said.
Making sure the special counsel investigation happens
Comey told Stephanopoulos that the need for a special counsel investigator was crystallized in Comey's mind because of the threat that Trump made to release alleged "tapes" of their conversations.
"When he tweeted that there might be tapes, the thought hit me like a lightning bolt, like, 'Wait a minute. If there are tapes, he will be heard on that tape in the Oval Office asking me to let it go. There is corroboration or could be corroboration for the thing we thought we'll never be able to corroborate,'" he said.
The "thing" Comey thought was unobtainable was evidence "of possible obstruction of justice."
"Somebody's gotta go get those tapes," Comey remembered telling himself, and he hoped that would be a special counsel "because I wanted someone without political conflict, someone to find the facts."
What would happen if Mueller gets fired?
Robert Mueller, the man who held the title of FBI director before Comey, was the one selected. It has been regularly speculated that Trump might try to fire Mueller, through Deputy Attorney General Rod Rosenstein, and Comey warns against that.
"It would, I hope, set off alarm bells that this is his most serious attack yet on the rule of law. And it would be something that our entire country -- again, Democrats and Republicans, that is higher than all the normal fights about policy. And it would be to the everlasting shame of partisans if they were unable to see that higher level and to protect it," Comey said.
Could the Russians have something on Trump?
As for what the special counsel might find, Comey isn't sure.
"Do you think the Russians have something on Donald Trump?" Stephanopoulos asked Comey.
"I think it's possible. I don’t know,” Comey responded. “These are more words I never thought I'd utter about a president of the United States, but it's possible."
What makes Trump "unfit" for office, according to Comey
Comey dismissed claims made by some that Trump is medically unfit to hold office.
"But not in the way ... I often hear people talk about it. I don't buy this stuff about him being mentally incompetent or early stages of dementia. He strikes me as a person of above average intelligence who's tracking conversations and knows what's going on," Comey said. "I don't think he's medically unfit to be president. I think he's morally unfit to be president."
"A person who sees moral equivalence in Charlottesville, who talks about and treats women like they're pieces of meat, who lies constantly about matters big and small and insists the American people believe it, that person's not fit to be president of the United States, on moral grounds.
"Our president must embody respect and adhere to the values that are at the core of this country. The most important being truth. This president is not able to do that. He is morally unfit to be president," he said.