The Minnesota Democrat said the hearing scapegoats and stereotypes Muslims, and will increase suspicion of the Muslim community, in turn making all Americans less safe.
"Violent extremism is a serious concern to all Americans and is the legitimate business of this committee," said Ellison, who testified at the House Committee on Homeland Security hearing today, but the approach "is contrary to the best of American values and threatens our security."
King's hearing, which he promised will be first in a series on the topic, has touched a nerve.
Democrats say the GOP leader is intentionally isolating Muslims, and is using religion to divide Americans.
Some members of the Muslim community fear that King is on a witch hunt, holding hearings that recall the days of Joseph McCarthy's anti-Communism crusade.
Many are concerned that it could "stoke a climate of distrust and fear in the Muslim community," Rep. Bennie Thompson, D-Miss., said. "I cannot help but wonder how propaganda about this hearing's focus on the American Muslim community will be used by those who seek to inspire a new generation of suicide bombers."
Members of the Muslim community believe they are being unfairly maligned because of the actions of a few. And Muslim leaders have pointed out that a number of terrorism suspects were arrested based on tips from their community.
Some say they are besieged and point to last year's stabbing of a New York Muslim cab driver and the controversy surrounding the scheduled burning of the Koran as recent examples of intolerance.
But King has said the goal of the House Committee on Homeland Security is not to target all Muslims but to deal with the reality of terrorism.
There's "nothing radical or un-American" about this hearing, King said in a staunch defense of his hearing.
"I remain convinced that these hearings must go forward and they will," King said in his opening statement. "To back down would be a craven surrender to political correctness and an abdication of what I believe to be the main responsibility of this committee; to protect America from a terrorist attack."
King said that as the 10-year anniversary of the 9/11 terrorist attacks approaches, Congress "must not let the memories of that tragic day to fade away.
"We must be fully aware that homegrown radicalization is part of al Qaeda's strategy to continue attacking the United States," King added. "Al Qaeda is actively targeting the American Muslim community for recruitment."
Melvin Bledsoe, whose son is charged with killing an Army private at a recruiting station in Arkansas, testified that his young son was converted and brainwashed by Islamic extremists.
"He was with true evil doers," he said. "We must stop these extremists from raping the minds of American citizens."
The hearing, filled with protesters and supporters, was often contentious as Democrats pounced on the panelists and challenged the Republicans' assertions.