The group of former CIA directors, CIA deputy directors and Director of National Intelligence called the move "ill-considered" and said the threat of additional removals are not based on security concerns but have "everything to do with an attempt to stifle free speech."
Brennan has drawn President Trump's ire for his vocal criticism of the President and his policies.
White House Press Secretary Sarah Sanders also said President Trump was considering removing the national security clearances of other former intelligence officials who have been critical of his presidency.
The move has united a who's who of former top intelligence officials who served under both Democratic and Republican administrations.
The list of former senior intelligence officials who signed the letter includes James Clapper, a former Director of National Intelligence, and seven CIA Directors including Robert Gates, William Webster, George Tenet, Porter Goss, Michael Hayden, Leon Panetta and David Petraeus.
"As former senior intelligence officials, we feel compelled to respond in the wake of the ill-considered and unprecedented remarks and actions by the White House regarding the removal of John Brennan’s security clearances," they wrote.
"We know John to be an enormously talented, capable, and patriotic individual who devoted his adult life to the service of this nation," the letter continued.
"Insinuations and allegations of wrongdoing on the part of Brennan while in office are baseless," it added.
Some of the officials who signed the letter noted that they may not side with Brennan's comments or choose to go public with any comments, but "We all agree that the president’s action regarding John Brennan and the threats of similar action against other former officials has nothing to do with who should and should not hold security clearances – and everything to do with an attempt to stifle free speech."
The officials said they had "never before seen the approval or removal of security clearances used as a political tool, as was done in this case."
They labeled the removal of Brennan's security clearance as "a signal" to current and former officials about potential limits to their free speech.
"As individuals who have cherished and helped preserve the right of Americans to free speech – even when that right has been used to criticize us – that signal is inappropriate and deeply regrettable," they wrote. "Decisions on security clearances should be based on national security concerns and not political views."
Another statement signed by 60 former CIA officials also supported Brennan and the right of former government officials to express their views "without fear of being punished for doing so."
Their letter said it was their "firm belief that the country will be weakened if there is a political litmus test applied before seasoned experts are allowed to share their views.was released on Friday."
Brennan received a separate endorsement on Thursday from retired Admiral William McRaven, the former head of U.S. Special Operations Command who oversaw the 2011 raid that killed Osama bin Laden.
In an open letter published in the Washington Post, McRaven expressed his support for Brennan and challenged the administration to also remove his security clearance.
"Few Americans have done more to protect this country than John. He is a man of unparalleled integrity, whose honesty and character have never been in question, except by those who don’t know him," McRaven wrote in his letter. "Therefore, I would consider it an honor if you would revoke my security clearance as well, so I can add my name to the list of men and women who have spoken up against your presidency."
In his letter McRaven also directly criticized President Trump saying "through your actions, you have embarrassed us in the eyes of our children, humiliated us on the world stage and, worst of all, divided us as a nation." He added, If you think for a moment that your McCarthy-era tactics will suppress the voices of criticism, you are sadly mistaken. The criticism will continue until you become the leader we prayed you would be."
Departing for New York on Friday, President Trump said he did not know McRaven, but added that he had received a "tremendous response from having done that because security clearances are very important to me."
Trump also doubled down on his criticism of Brennan saying "I've never respected him" and said he would likely soon remove the security clearance of Justice Department official Bruce Ohr.