Since regulators slapped the bank with a $185 million fine on Sept. 8, Stumpf has faced near constant calls to step down and has been subject to two bruising congressional hearings.
In a statement released by the company, Stumpf said: "While I have been deeply committed and focused on managing the Company through this period, I have decided it is best for the Company that I step aside."
The bank's board, of which Stumpf was the chairman, elected Tim Sloan who had been the company for 29 years and most recently served as the company's Chief Operating Officer and President to replace the outgoing CEO immediately, the bank said.
Meanwhile, the bank's Lead Director, Stephen Sanger, has become the board's non-executive chairman effective immediately.
In the statement, Sloan said that his "immediate and highest priority is to restore trust in Wells Fargo."
"We're not going to allow the last five weeks to define the future success of a 164-year-old company," Sloan said during an interview with CNBC shortly after the announcement.
Stumpf will not be receiving a severance package, a company official and another person who was familiar with the matter told ABC News.
One familiar source said that Stumpf would have access to retirement benefits after six months. The source did not provide details of those benefits.
The shake-up at the top ranks of the bank comes as the company's reputation has taken a battering for weeks in the press and the halls of government.
A barrage of criticism has been directed at Stumpf and the bank since the Consumer Financial Protection Bureau (CFPB), the Los Angeles City Attorney and the Office of the Comptroller of the Currency made explosive allegations on Sept. 8 that -- according to the CFPB -- Wells Fargo employees had opened or attempted to open more than 2 million deposit or credit card accounts without customers' permission between May 2011 and July 2015.
Some 5,300 employees were fired over that time period, the bank confirmed, while never fully admitting to wrongdoing; saying instead that it regrets and takes responsibility "for any instances where customers may have received a product that they did not request."
For his part, Stumpf has careened between blaming employees and accepting "full responsibility for all unethical sales practices."
To squelch the fallout, the company quickly announced that by year's end it would be discontinuing the sales goals program imposed on employees that was pinned as a coercive by critics.
If the company had hoped that ending the program would have released some pressure on the company they were wrong.
In less than two weeks of the allegations being leveraged against it, the FBI and federal prosecutors in New York and California began probing the bank, the House Financial Services committee launched an investigation and Stumpf faced blistering questions in front of the Senate Banking Committee.
It was in Banking Committee's hearing that he faced calls to resign by Sen. Elizabeth Warren, D-Massachusetts, and was told by Sen. Bob Corker, R-Tennessee, that he would be engaging in malpractice if the board did not claw back money it had previously paid executive during the period the accounts were allegedly being opened.
As September dragged on, lawsuits were filed by customers, employees and shareholders. Various senators asked the Department of Labor to investigate the bank over its labor practices and, separately, slammed the bank for its clauses in its customer contracts that they said restricted customers' ability to sound the alarm. And Stumpf visited Washington once more, facing a four-plus hour bipartisan barrage of questioning in front of the House Financial Services Committee.
“Fraud is fraud. Theft is theft. And what happened at Wells Fargo over the course of many years cannot be described any other way,” Stumpf was told by the the committee's chairman, Jeb Hensarling, R-Texas, at the outset.
Giving into pressure, Wells Fargo's board announced late on Sept. 27 that Stumpf would forgo $41 million worth of promised compensations and that his salary would be suspended while it launched an investigation.
Meanwhile, the banks shares continued to shed value -- ending the month down over 11 percent from the day when the scandal erupted.
October brought little relief for the company, with fourteen Senators calling on the Justice Department to investigate the bank's top brass, as ABC News first reported.
Despite the Congress and the Senate adjourning ahead of the election, pressure remained on the 164-year-old bank with regular press reports bringing out new allegations against the company.
It isn't yet clear what was the final tipping point, but shortly after markets closed on Wednesday word came from the bank that Stumpf was out.
At one point the company's stock bounced more than 2.1% in after hours trading on the news.
Stumpf, who had been with the company for 34 years, had been CEO since June 2007 and chairman of the board since January 2010.