While others in the White House, like senior adviser David Axelrod, are the voice of the liberal conscience, Emanuel was often the brass tacks pragmatist. Many progressives and liberals held him responsible for not pushing harder -- or at all -- for a public option in the health care legislation, or for criminal trials for Guantanamo defendants linked to the 9/11 attacks.
Known for his brash personality, Emanuel began each day before the sun, often by swimming a mile, and was perhaps the hardest worker in the White House, a requirement for the all-consuming job of chief of staff.
White House press secretary Robert Gibbs this month joked about one of Emanuel's favorite phrases, summing up his work ethic during his time at the White House.
"As Rahm sometimes jokes," Gibbs recalled, "if it's Saturday, there's just two more full workdays until Monday."
Emanuel's likely departure comes at a critical time for the White House and for Obama. Just five weeks before the crucial midterm elections when Democrats in races across the country face possible defeat, the administration is also balancing tough poll numbers.
The most recent ABC/Washington Post poll from early September shows the president's approval ratings at 46 percent, with a 52 percent disapproval rate.
A new incoming chief of staff could bring in energy and fresh ideas, and help the White House retool operations, as the administration picks up the pieces of what could be a difficult election night for Democrats.
Obama is known for his loyalty, maintaining a tight inner circle, and is not liking to bring outsiders into his administration, so it is likely that the next pick for chief of staff would come from someone with experience inside the administration and a relationship with the president.
Vice President Biden's chief of staff Ron Klain, Deputy National Security Advisor Tom Donilon and White House deputy chief of staff Jim Messina are all said to be on the list of candidates from within the president's inner circle and already working at 1600 Pennsylvania Avenue.
Yet the choice may also be made to appoint someone from outside the administration to capitalize on some fresh blood and new ideas within the administration.
Former Senator Majority Leader Tom Daschle and Obama's 2008 campaign manager, David Plouffe, are both said to be on the list of names to be considered from outside the administration.
Emanuel's departure is just of many departures the White House has faced recently, including many from the president's economic team.
Last week the White House announced that National Economic Council director Larry Summers is expected to leave the administration after the mid-term elections.
Earlier this month Christina Romer resigned as the chairwoman of the President's Council of Economic Advisors.
And Office of Management and Budget director Peter Orzsag resigned over the summer as well.
The White House has been working hard to downplay the departures.
"I think there's no doubt that there will be people that return to their lives and their families," Gibbs said of the typical turnover that occurs within presidential administrations. "But we've got a while before that. We've got at least two months before this election -- or about two months before this election before we get to a lot of those decisions."