Since regulators slapped the bank with a $185 million fine on Sept. 8, Stumpf has faced near constant calls to step down and has been subject to two bruising congressional hearings.
In a statement released by the company, Stumpf said: "While I have been deeply committed and focused on managing the Company through this period, I have decided it is best for the Company that I step aside."
The bank's board, of which Stumpf was the chairman, elected Tim Sloan who had been the company for 29 years and most recently served as the company's Chief Operating Officer and President to replace the outgoing CEO immediately, the bank said.
Meanwhile, the bank's Lead Director, Stephen Sanger, has become the board's non-executive chairman effective immediately.
In the statement, Sloan said that his "immediate and highest priority is to restore trust in Wells Fargo."
"We're not going to allow the last five weeks to define the future success of a 164-year-old company," Sloan said during an interview with CNBC shortly after the announcement.
Stumpf will not be receiving a severance package, a company official and another person who was familiar with the matter told ABC News.
One familiar source said that Stumpf would have access to retirement benefits after six months. The source did not provide details of those benefits.
The shake-up at the top ranks of the bank comes as the company's reputation has taken a battering for weeks in the press and the halls of government.
Some 5,300 employees were fired over that time period, the bank confirmed, while never fully admitting to wrongdoing; saying instead that it regrets and takes responsibility "for any instances where customers may have received a product that they did not request."
For his part, Stumpf has careened between blaming employees and accepting "full responsibility for all unethical sales practices."
To squelch the fallout, the company quickly announced that by year's end it would be discontinuing the sales goals program imposed on employees that was pinned as a coercive by critics.
If the company had hoped that ending the program would have released some pressure on the company they were wrong.
“Fraud is fraud. Theft is theft. And what happened at Wells Fargo over the course of many years cannot be described any other way,” Stumpf was told by the the committee's chairman, Jeb Hensarling, R-Texas, at the outset.
Giving into pressure, Wells Fargo's board announced late on Sept. 27 that Stumpf would forgo $41 million worth of promised compensations and that his salary would be suspended while it launched an investigation.
Meanwhile, the banks shares continued to shed value -- ending the month down over 11 percent from the day when the scandal erupted.
October brought little relief for the company, with fourteen Senators calling on the Justice Department to investigate the bank's top brass, as ABC News first reported.
Despite the Congress and the Senate adjourning ahead of the election, pressure remained on the 164-year-old bank with regular press reports bringing out new allegations against the company.
It isn't yet clear what was the final tipping point, but shortly after markets closed on Wednesday word came from the bank that Stumpf was out.
At one point the company's stock bounced more than 2.1% in after hours trading on the news.
Stumpf, who had been with the company for 34 years, had been CEO since June 2007 and chairman of the board since January 2010.